Objectives Critical appraisal of the literature is an integral and important part of surgical practice, but can this skill be taught to young doctors? The purpose of this study was to investigate the effect of formal instruction regarding critical review and appraisal of journal articles, using junior surgical residents followed over the course of a 10-week long programme.

Methods First-year surgical residents who participated in the department of orthopaedic surgery's compulsory journal club evaluated one article per week for 10 weeks, using the reviewer guidelines and a scoring system currently used by Arthroscopy. The article was selected by a senior consultant orthopaedic surgeon and was provided for assessment to each resident the week prior. The scores and evaluation recommendation (accept, revise and reject) of the residents were then compared against the senior surgeon's assessment. A contingency table and Fisher's exact test was used to compare the frequency of agreement between the decisions of the senior surgeon and the residents.

Results Twenty residents were included. Agreement in overall total assessment scores increased significantly (p=0.0001) from 49.5% at session 1 to 82.5% at session 10. When comparing the mean percentage of agreement of the first five sessions (61%) with the second five sessions (95%), a significant (p=0.03) increase was observed. The percentage of agreement with the senior surgeon (whether the article should be accepted, revised or rejected) improved from 0% for the first session to 60% at the last session (χ²=7.2–11.2, p=0.02–0.04).

Conclusions The results strongly suggest that a structured approach to the review and appraisal of journal articles using the format of a journal club significantly improves critical reading skills for first-year surgical residents.