Article Text

PDF

Improving medical students’ written communication skills: design and evaluation of an educational curriculum
  1. L Melvin1,
  2. K Connolly1,
  3. L Pitre2,
  4. K L Dore2,
  5. P Wasi2
  1. 1Department of Medicine, University of Toronto, Toronto, Ontario, Canada
  2. 2Department of Medicine, McMaster University, Hamilton, Ontario, Canada
  1. Correspondence to Dr Lindsay Melvin, Department of Medicine, Toronto Western Hospital, University of Toronto, 399 Bathurst Street, 8EW-432E, Toronto, Ontario, Canada M5G 2S8; Lindsay.melvin{at}rogers.com

Abstract

Background and objectives Written and verbal communication skills are important skills for all physicians. While verbal skills are taught and assessed in medical school, medical students report limited instruction in written communication skills. This study examined the impact of a curriculum delivered during a 6-week clinical rotation in Internal Medicine on the objective assessment of medical students’ written communication skills.

Methods The curriculum consisted of two educational programmes: a medical student communication tutorial and a resident feedback workshop. The study was conducted from March 2012 to January 2013 at McMaster University in Hamilton, Ontario, Canada. The study featured three arms: (1) control, (2) medical student communication tutorial alone and (3) student tutorial and resident feedback workshop. Data were collected on 126 students during 6-week Internal Medicine clerkship rotations. Students’ written consultation notes were collected prior to the educational programmes and at 6 weeks. Blinded faculty assessors used an independently validated Assessment Checklist to evaluate consultation notes.

Results Consultation note scores improved from week 1 to week 6 across all study arms. However, the change was statistically significant only in arm 3, featuring both the medical student tutorial and the resident feedback workshop, with mean scores improving from 4.75 (SD=1.496) to 5.56 (SD=0.984) out of 7. The mean difference between week 1 and week 6 was significantly different (0.806, p=0.002, 95% CI 0.306 to 1.058).

Conclusions The combination of a resident feedback workshop with medical student written communication tutorial improves objective evaluations of consultation note scores over student tutorial alone.

  • EDUCATION & TRAINING (see Medical Education & Training)
  • GENERAL MEDICINE (see Internal Medicine)
  • MEDICAL EDUCATION & TRAINING

Statistics from Altmetric.com

Introduction

Deficiencies in written and verbal communication within a healthcare team have been directly linked to increased incidence of patient-related errors.1 Despite the critical nature of these competencies in practice, the teaching of communication skills to medical students is often informal, either through an apprenticeship model without formal feedback or through the hidden curriculum.2 The literature addressing communication skills in medical education often focuses on verbal communication between the physician and the patient. Few studies address the important facets of communication that involve the synthesis of historical information into written admission notes, leading to a paucity of information on best practices. Written documentation of an initial patient encounter, referred to as a consultation notes, represents a complex, multifaceted task in which a student must collect, organise and synthesise a wealth of information from a patient interaction.3–6 The written case presentation is a fundamental skill across all disciplines in medicine and a necessary competency for students to achieve in medical training.7 ,8

It has been previously reported that medical students largely learn communication skills through trial and error, and that they rely heavily on feedback from residents and staff physicians to refine their abilities.3 ,9 A key setting for developing these skills is during on-call shifts, where students communicate directly with residents and staff about specific patient encounters. The feedback and instruction that students receive on their written documentation during and immediately following a call shift are highly dependent on several factors, including the available time for supervising residents or staff to give directed instruction, the resident or staff comfort with delivering constructive feedback, as well as staff and resident preferences as to what written documentation should include. Such variables result in a heterogeneous mixture of medical student competency and comfort with oral and written communication at the end of rotations.10

Many medical schools in Canada attempt to facilitate feedback on communication skills using encounter cards. Encounter cards are portable evaluation tools for a specific skill (eg, history taking, physical examination (PE), etc) and are used following the assessor observing the student in a clinical encounter. Encounter cards have permitted some assessment of communication abilities but their use has not been shown to result in improvement of medical student's oral presentation or written communication skills.11 The Reporter, Interpreter, Manager, and Educator (RIME) model for delivering feedback, designed by Pangaro and colleagues, is the most well-studied assessment system for junior trainees, including both medical students and early residents. Though intended for oral case presentation feedback, the model can be adapted to provide feedback on written communication skills, as the structure is similar. The RIME model allows the feedback provider to stratify learners into categories on the learning continuum, providing a framework for the student to gauge their progress and set clear goals for improvement.12 Studies have demonstrated that ratings assigned by senior residents and faculty using RIME were predictive of students’ final examination performance in clerkship placements, both in Internal Medicine and in Emergency Medicine. Interestingly, the residents’ scores of students were most predictive of a student's final examination performance in Internal Medicine, highlighting the fact that senior residents may be best able to deliver accurate feedback relevant to the learner, perhaps due to proximity of training and increased face time.13–15 The RIME model is easily translated in the clinical context and facilitates delivery of real-time, learner-centred feedback on clinical performance.16 Developing a process of structured feedback on written communication skills could result in both global consistency and individual improvements in written communication.

The importance of instruction in written communication in medical education has been well described in the literature, yet interventions designed to produce concrete improvements in written communication skills have not been well studied. There exists no national or international consensus on the specific constructs defining an adequate or exemplary consultation note,8 limiting the ability to provide meaningful feedback or objective assessment of these skills.17 To assess communication skill training in our own institution and to inform our study, we administered an electronic needs assessment survey to 90 students who had completed their clerkship rotation in Internal Medicine at McMaster University. The survey focused on the students’ experiences with teaching and feedback on communication skills during their Internal Medicine rotation. Of the students contacted, 58/90 (64%) completed the survey. Of the students, 53/58 (91%) indicated that they would have liked more instruction on how to document an Internal Medicine consult note.

Our study objective was to improve the quality of medical student written communication skills, using the consultation note, by way of increasing instruction and feedback that medical students receive on these skills. We hypothesised that delivering educational programmes would objectively improve the quality of student written communication skills.

Methods

Study design

The prospective, partially masked controlled trial was run for six consecutive Internal Medicine clerkship rotations, each lasting 6 weeks, from March 2012 to January 2013. We devised two educational programmes: one for medical students on communication skills (box 1) and one for senior residents on feedback specifically addressing communication skills (box 2). The study had three arms: control (arm 1), student communication tutorial only (arm 2) and student communication tutorial with senior resident feedback workshop (arm 3) (figure 1). To avoid contamination, study arms were separated by hospital site. A study assistant collected written consultation notes by the students on newly admitted patients. Notes were collected during the first week of the students’ rotation, prior to the delivery of the educational programme, and again in week 6, the final week of their rotation. The study assistant removed all student and patient identifiers. The written consultation notes were scored using the independently validated Assessment Checklist (see below).18

Box 1

Medical student tutorial

The medical student communication tutorial was developed by study authors and was informed by literature review and discussion with local medical education experts. The tutorial was delivered to the medical students in the intervention arms at the end of the first week of their rotation by study authors LM and KC. The tutorial included an interactive presentation highlighting the key components of the Internal Medicine oral and written case presentations. The tutorial included a video demonstration and break-out session where students evaluated written consultation notes and highlighted areas for improvement.

Box 2

Resident feedback workshop

The resident feedback workshop was also developed by study authors and used an interactive presentation to introduce methods of delivering effective feedback to clerks on written communication. It was delivered to the senior medical residents by study authors LP and PW during the same week the medical students received their tutorial. To enable continuous reinforcement and feedback, the study authors developed a Reporter, Interpreter, Manager, and Educator (RIME)-based Feedback Tool (see online supplementary appendix 1) designed for use on-call specifically targeted to communication skills. The workshop highlighted the use of RIME as a feedback mechanism for both the oral presentation, as described in the literature, and expanded upon its use for written communication skills. The tool also featured checkboxes for scoring of the clinical encounter level of difficulty of ‘Straightforward’, ‘Difficult’ or ‘Very Difficult’ to address the reality that not all encounters, and thus not all communications, are equal in their level of complexity. Details of the tool were provided on cards and provided for the residents in the on-call workrooms of the hospital.

Participants and setting

The study participants were 126 medical students on the Internal Medicine clinical clerkship rotation and 13 senior medical residents (box 3). The study was conducted at the Hamilton General Hospital, St. Joseph's Healthcare Hamilton and the Juravinski Hospital, teaching hospitals affiliated with McMaster University in Hamilton, Ontario, Canada (box 4).

Box 3

Canadian medical training

In the Canadian medical education system, medical students spend up to 2 years in the classroom followed by 2 years of hands-on clinical training, referred to as ‘clinical clerkship’, wherein students rotate through various clinical rotations, including specialties such as Surgery, Paediatrics and Internal Medicine. Canadian medical students are often given significant autonomy and are expected to work ‘on-call’ shifts with residents overnight in the hospital. The students are typically responsible for performing the initial clinical assessment of newly admitted patients and presenting their findings to residents and staff. Medical residents are junior doctors who have committed to a specialised field and who take on a supervisory and educational role with the medical students. Staff physicians oversee the care of patients as undertaken by the medical students and residents.

Box 4

Setting

The study was conducted at McMaster University in Hamilton, Ontario, Canada. Medical students are assigned to one of three teaching hospitals during their 6-week rotation in Internal Medicine. The rotation is standardised across the three sites so the training students receive is highly comparable with similar patient mix and volume. Study arms were separated by these hospital sites. At McMaster University, the medical students rotate through different specialties in clerkship throughout their second and third year of medical training in variable sequence, which means student experience levels vary at the start of each Internal Medicine rotation.

Instrument design: Written Consultation Note Assessment Checklist

An Assessment Checklist to evaluate medical student consultation notes was developed after thorough literature review followed by consensus of a focus group consisting of expert faculty and senior residents. The checklist was divided into subsections of common written consultation note headings (history, PE, assessment and plan), as well as handwriting, and consisted of dichotomous Yes/No questions and questions scored on a 7-point Likert Scale. The scale was anchored at 1 and 7 by ‘Disagree’ and ‘Agree’. Each subsection had a Global Score assigned based on a 7-point Likert Scale ranging from ‘Poor’ to ‘Excellent’, and a Global Score on the same 7-point scale was assigned to the note as a whole.

The Assessment Checklist was validated using 50 randomly selected medical student consultation notes from the first 100 notes collected.18 The notes were scored by two raters, a combination of five residents and five staff from the Internal Medicine department. The internal consistency of the checklist was assessed using Cronbach's alpha and found to be 0.79 (95% lower confidence limit 0.73). The inter-rater reliability was calculated for the global consultation note score between all pairs of markers using a linear-weighted kappa score and was calculated to be 0.69. Raters completed a survey to determine the acceptability of the Assessment Checklist; 8/10 (80%) surveyed found the checklist somewhat easy or very easy to use. Through this process, we determined that the Assessment Checklist provided a portable and reliable method to evaluate the medical student consultation notes for our study.

Data collection

The study assistant collected copies of consultation notes completed by the medical students on call the previous night during week 1 and week 6 of their Internal Medicine rotations from March 2012 to January 2013. Medical student consultation notes were identified via signatures. Student and patient information was removed from the notes, and then consultation notes were pooled. Masked faculty assessors not involved in the study or student evaluation performed consultation note scoring. Assessment Checklist scores were entered into a database.

Data analysis

Scores from the Assessment Checklist were analysed from week 1 to week 6 within study arms. Scores for each of the consultation note subsections (history, PE, assessment and plan), as well as overall scores for the entire consultation note, were analysed between week 1 and week 6. Analyses of improvement were conducted as an aggregate score of week 1 and week 6 students; pre-performance and post-performance of individual students were not studied as identifying student information was removed. Analyses were performed to determine correlations between handwriting scores and subsection scores with the overall global scores.

Statistical analysis

Global consultation note scores were designated as the primary outcome. An independent t test was used to compare scores between weeks 1 and 6 for each arm of the study. For the changes of global scores between arms, a generalised linear regression model was used, using pre-grouping–post-grouping and treatment arm as the independent variables. Pearson correlation was used to study associations between handwriting and subsection scores (history, PE and assessment and plan) with the overall global score. As no previous studies were available to inform this area, no sample size calculation was performed. Statistical analyses were performed by SPSS (V.20th, IBM Corp, Armonk, New York, USA).

Results

Participants

The study enrolled 126 medical students from McMaster University over the course of the six consecutive Internal Medicine clerkship rotations. A total of 313 consultation notes were collected. Thirteen senior Internal Medicine residents participated in the resident feedback workshop (arm 3).

Global written consultation note scores

The mean global consultation note scores improved across all study arms from week 1 to week 6 (table 1). The measured difference was greatest, and only achieved statistical significance, in study arm 3 featuring the student tutorial and resident workshop combined. In the control arm (arm 1), the mean global score (SD) from week 1 was 4.61 (1.214) and increased to 5.03 (0.972) in week 6. The mean difference (95% CI) of 0.420 (0.037 to 0.878, p=0.071) was not significant. In the student tutorial arm (arm 2), the mean global score in week 1 was 4.77 (0.824) and in week 6 was 5.09 (1.18); the mean difference was 0.316 (0.073 to 0.704, p=0.110). In the arm with the student tutorial and resident workshop (arm 3), the mean global score in week 1 was 4.75 (1.496) and in week 6 was 5.56 (0.984). The mean difference between the weeks was 0.806 (0.306 to 1.058, p=0.002). The difference in the change in scores from weeks 1 to 6 did not meet statistical significance between arm 2 and arm 1 (p=0.52), but did meet significance between arm 3 and arm 1 (p=0.034). Global scores ranged from 1 to 7 in all arms and weeks of the study demonstrating appropriate use of the scale.

Table 1

Global written consultation note scores

Consultation note subsection scores

Scores were also assigned for subsections of the written consultation notes: the history, including both past medical history and the history of present illness (HPI), PE and assessment and plan subsections (A&P) (table 2).

Table 2

Subsection global scores

In the control arm (arm 1), the PE subsection scores improved with a mean score difference (95% CI) of 0.56 (0.58 to 1.051, p=0.029) between weeks 1 and 6. The A&P subsection scores also improved over the study duration, with a mean difference of 0.62 (0.101 to 1.125, p=0.19).

In the student tutorial alone arm (arm 2), the A&P subsection improved over the study duration, with mean score difference of 0.53 (0.048 to 0.999, p=0.031).

In the arm with student tutorial and resident workshop (arm 3), HPI subsection scores showed mean difference of 0.56 (0.062 to 1.058, p=0.028), PE subsection scores improved by 0.65 (0.214 to 1.091, p=0.004) and A&P subsection scores by 0.73 (0.192 to 1.265, p=0.008).

Correlations between subsection scores and handwriting scores with global written consultation note scores

Pearson correlations were performed between subsection scores (HPI, PE, A&P). Correlation coefficients (R) are reported, while the values between 0.7 and 1.0 and values between 0.3 and 0.7 indicate a strong linear relationship and moderate linear relationship, respectively. Correlations data revealed that the subsection score for HPI correlated significantly with the global written consultation note score, R=0.728 (p<0.0001, N=310), indicating a strong correlation. There was moderate correlation with subsection PE scores with the global written consultation note score, R=0.667 (p<0.0001, N=311). The strongest correlation with the global written consultation note score was with the subsection A&P scores, with R=0.806 (p<0.0001, N=309). Handwriting scores were significantly associated with the global consultation note score, with R=0.441 (p<0.0001, N=310).

Discussion

We have demonstrated that the implementation of an educational programme for medical students on written communication skills, when coupled with a programme for residents on feedback techniques specifically addressing communication skills, results in measurable improvement in the quality of medical students’ written consultation notes. These findings emphasise the importance of equipping supervisors, specifically residents, with skills needed to provide effective feedback, in addition to student-targeted educational interventions.

The improvement demonstrated in consultation note scores for the history, PE and assessment and plan subsections as well as the global consultation note scores in study arm 3, featuring both the medical student communication tutorial and the resident feedback workshop, indicates a positive interaction between the residents and medical students. In comparison with study arm 2, which lacked resident intervention, we can infer that feedback from senior residents is an important factor in improving written communication skills for medical students. One of the well-described difficulties for faculty when it comes to delivering feedback, but no less important for residents, is the challenge of providing constructive, relevant feedback in a busy clinical setting.19 The use of the RIME model facilitates and simplifies constructive feedback delivery and assists in the refining of a medical student's skills. While the study design did not allow for specific observation of residents giving feedback, the results show that a workshop equipping residents with tools for effective feedback using the RIME model facilitated improvement in medical student consultation note quality.

In addition to the value of feedback in written communication skill development, conclusions about communication skill development can be drawn from the correlations found among the subsection and handwriting scores with the global consultation note scores. The strongest correlation with global consultation note scores was with scores assigned for the Assessment and Plan subsection. The ‘Assessment and Plan’ subsection of a consultation note demonstrates a student's ability to highlight relevant features of the history and PE and use of clinical reasoning to generate a management plan. A strong Assessment and Plan demonstrates abilities in several domains, including history taking, organisation and cognitive reasoning skills. Perhaps unsurprisingly, this section of the note is highly valued when assessed by faculty. Additionally, the handwriting score correlated well with the global consultation note score. It is important for students to be aware that handwriting, an aspect unrelated to their medical knowledge or clinical reasoning capabilities, impacts the overall assessment of their communication skills. Handwriting is a well-recognised potential source for medical error and miscommunication, but to our knowledge, we are the first to demonstrate a direct correlation with medical student assessment. The relative importance of the Assessment and Plan, as well as handwriting, on the overall score for the consultation note indicates the value of these areas for assessment of the trainee.

The assessment and final evaluation of a medical student's performance incorporates many daily activities, including but not limited to the written consultation note, with an overall gestalt about the student's capabilities. The creation and organisation of a written consultation note is an essential skill for students in medical education and for faculty in assessment of students.8 ,10 The written consultation note can provide information about the students’ ability to collect, organise and synthesise complex patient information. Implementing the curriculum for all medical students reduces heterogeneity in foundational knowledge and informs students of performance expectations. Ensuring all students have the same baseline preparation in written communication skills is an important step towards student advancement of these skills and equitable assessment by faculty raters.

Formal evaluation of students also requires objective and reliable tools. Through this study, we developed and validated an Assessment Checklist to formatively assess medical student consultation notes which has utility in both educational and research endeavours, as it provides a standardised method of assessment of a clinical competence required for medical students and junior residents. Similar to Kogan and Shea, we found that using both dichotomous and Likert Scale questions facilitated assessors in coming to a final global rating of the written consultation note.17 We feel, however, that including details in the Assessment Checklist, rather than simply global ratings, allows the rater to subsequently give specific examples for improvement to the learner in real time, when implementing the checklist. The checklist is easy to use and requires no advanced training, facilitating implementation in both research realms and daily clinical practice.

Our study has several limitations. We did not perform a sample size calculation as we did not have any baseline information available, as such, we collected as many consultation notes as possible in our study period. The separation by hospital site, though designed to reduce contamination, may have confounded the results. Furthermore, the improvements seen in consultation note scores raise the important question of the meaningful clinical impact of the improvement and whether the improvement is demonstrative of attaining competency in this skill, or correlates with another important outcome such as examination scores or final evaluations. As we chose to keep individual student results anonymous, and did not couple them to final examination grades or evaluations from the core Internal Medicine rotation, we were not able to investigate these associations. Future studies are needed to demonstrate if scores in written communication skills correlate with final rotation evaluations or other standardised measures of evaluation of medical student clinical competence.

Postscript

Since study completion at McMaster University, the resident feedback workshop has been incorporated into the senior resident curriculum. The medical student tutorial has been made available for use by clinical teachers at their discretion. The Assessment Checklist has been incorporated in subspecialty clerkship rotations and is available for use on Internal Medicine core clerkship rotations.

Main messages

  • Written documentation of patient encounters is a necessary skill in all medical disciplines.

  • Teaching and evaluation of medical student written communication skills occurs mainly through clinical experience on rotations.

  • Educational interventions directed at medical students and senior residents can objectively improve medical students’ written communication skills.

Current research questions

  • Are the results of this study replicable in settings outside of Internal Medicine, or are the findings specialty specific?

  • Does improvement in written communication correlate with other markers of medical student evaluation such as performance in examinations?

  • What are the necessary requirements to deem a student ‘competent’ in written communication?

Key References

  • Wiese J, Varosy P, Tierney L. Improving oral presentation skills with a clinical reasoning curriculum: a prospective controlled study. Am J Med 2002;112:212–18.

  • Pangaro L. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med 1999;74:1203–7.

  • Lingard LA, Haber RJ. What do we mean by “relevance”? A clinical and rhetorical definition with implications for teaching and learning the case-presentation format. Acad Med 1999;74(10 Suppl):S124–7.

  • Ratcliffe TA, Hanson JL, Hemmer PA, et al. The required written history and physical is alive, but not entirely well, in internal medicine clerkships. Teach Learn Med 2013;25:10–14.

  • Griffith CH 3rd, Wilson JF. The association of student examination performance with faculty and resident ratings using a modified RIME process. J Gen Intern Med 2008;23:1020–3.

Acknowledgments

The authors would like to thank Carol Mantle for her invaluable assistance and Yuhong Yuan and Ji Cheng for their statistical guidance.

References

View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Twitter Follow Lindsay Melvin at @LMelvinMD

  • Contributors LM, KC, LP and PW all contributed to the study design, execution, data collection, analysis and creation and revision of the manuscript. KLD provided expert guidance in the methodology as well as manuscript revision. CM collected study data. CY and JC provided expert guidance in statistical methods and analysis.

  • Funding This study was funded by the Core Internal Medicine Medical Education Research Grant, awarded to the study authors by the Internal Medicine programme at McMaster University.

  • Competing interests None.

  • Ethics approval Ethics approval was sought and obtained from the Hamilton Integrated Research Ethics Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.