Article Text

PDF
Effect of rating scales on scores given to junior doctors in multi-source feedback
  1. Andrew Hassell1,4,
  2. Alison Bullock1,2,
  3. Andrew Whitehouse1,
  4. Lawrence Wood1,3,
  5. Peter Jones1,4,
  6. David Wall1
  1. 1Postgraduate Medical and Dental Education, NHS West Midlands Workforce Deanery, Birmingham, UK
  2. 2Department of Social Sciences, Cardiff University, Cardiff, UK
  3. 3Department of Obstetrics and Gynaecology, University Hospital of Coventry and Warwick, Coventry, UK
  4. 4Institute of Primary Care and Health Sciences, Keele University, Keele, UK
  1. Correspondence to Dr Andrew Hassell, School of Medicine, Keele University, Staffordshire ST5 5BG, UK; a.hassell{at}keele.ac.uk

Abstract

Background Multi-source feedback (MSF) has an established role in the workplace based assessment of doctors in training. Different models of MSF are currently used in different training programmes and settings. One important way in which these models differ is the rating scale on which assessors score the trainee. The aim of this study was to explore the effect of rating scale on MSF scores.

Methods Foundation Year 2 trainees in hospitals in the West Midlands underwent MSF using the validated MSF tool, team assessment of behaviour (TAB) in autumn 2005. Trainees were scored with TAB using one of four different rating scales, ranging from 3- to 9-point scales. Each participating hospital used only one rating scale. The proportions of trainees scored as having potential problems were related to the different rating scale used. Similarly, the proportions scored as ‘above expectations’ were compared. Assessors also completed a short questionnaire regarding the assessment.

Results 245 trainees underwent 2594 assessments. Longer rating scales were associated with a lower proportion of trainees awarded ‘problem’ scores and higher proportions of trainees scored as ‘above expectations’. Assessors generally reported no difficulties whichever rating scale they had used.

Conclusion Careful consideration, recognising its potential impact on assessment score, should be given to the rating scale used when instituting MSF within a system of workplace based assessment.

  • 360 degree assessment
  • multi-source feedback
  • medical education & training
  • rheumatology
  • medical education and training
  • education and training (see medical education and training)

Statistics from Altmetric.com

Footnotes

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.