Purpose of study Previous research demonstrates that graduating medical students often feel unprepared for practice and that their perceptions of preparedness correlate only partially with those of their supervising consultants. This study explores the components of preparedness for practice from the perspectives of both newly qualified doctors and their educational supervisors.
Study design A questionnaire study was undertaken at the University of Edinburgh, involving feedback on preparedness for practice over three consecutive years from 2007 to 2009, against 13 major programme outcomes, from graduates and their educational supervisors. In addition, free text responses were sought and thematically analysed.
Results Graduates consistently felt well prepared in consultation and communication skills but less prepared in acute care and prescribing. Educational supervisors consistently felt that graduates were well prepared in information technology and communication skills but less prepared in acute care and practical procedures. Free text analysis identified four main themes: knowledge; skills; personal attributes; and familiarity with the ward environment.
Conclusions Preparedness for practice data can be enriched by repeated collection over several years, comparison of different perspectives, and incorporation of free text responses. The non-technical skills of decision-making, initiative, prioritisation, and coping with stress are important components of preparing new doctors for practice. Education for Foundation trainees should focus on the areas in which graduates are perceived to be less prepared, such as acute care, prescribing, and procedural skills.
- clinical competence
- preparedness for practice
- medical education and training
Statistics from Altmetric.com
- clinical competence
- preparedness for practice
- medical education and training
One of the fundamental aims of any primary medical educational programme is to prepare students adequately for clinical practice. In the UK, most medical graduates proceed directly to the 2 year Foundation programme, which consists of 4 month hospital and community based rotations. Foundation year 1 doctors (FY1s) are closely supervised during the day, but also work out-of-hours shifts during which supervision is provided only when requested. Between 1993 and 2009, the General Medical Council has produced several versions of Tomorrow's Doctors, a document which provides guidance to medical schools on how best to ensure that their graduates are “properly prepared for clinical practice and the Foundation Programme”.1
Between 1998 and 2006, all UK medical schools initiated major curricular revisions to align their courses with the recommendations featured in the 1993 version of Tomorrow's Doctors.2 It was hoped that these changes would improve graduates' preparedness for practice. However, recent UK studies continue to demonstrate shortfalls in the perceived preparedness of graduates, with only 59% of 2004 graduates3 and 58% of 2005 graduates4 agreeing that their medical school had prepared them for the jobs undertaken when qualified.
Previous studies have demonstrated variable degrees of consistency between the ratings that newly qualified doctors give themselves and those afforded by their educational supervisors (ES). In a West Midlands study, graduates consistently rated themselves significantly higher than did their ES in the vast majority of domains.5 However, studies in Manchester6 and Liverpool7 8 demonstrated a better degree of concordance between the perceptions of newly qualified doctors and their supervisors. A Bristol study that asked new doctors near the end of their first postgraduate year to rate their own levels of competence in a variety of domains found no correlation whatsoever with matched ratings in the same domains provided by their supervising consultants.9
While several studies have examined both graduate and educational supervisor perspectives at a single point, none has done so over consecutive years. In addition, few studies undertaken to date have moved beyond the ratings of graduates and ES in predefined domains to ask what additional areas those groups perceive to be important in easing the transition to clinical practice.
The overarching aim of this study was to explore preparedness for practice from the perspectives of both newly qualified doctors and their ES. In order to achieve this, the study aimed to address a series of questions relating to preparedness for practice:
How well do medical graduates feel that their primary medical training prepared them for starting work as a doctor in a variety of predefined domains?
How well do ES of medical graduates feel that primary medical training prepared those graduates for starting work as a doctor in a variety of predefined domains?
How do medical graduates' perceptions of their preparedness for practice in predefined domains compare with those of ES?
Which additional areas do graduates and/or ES identify as important in preparation for practice?
This study was undertaken in the context of the MBChB course at the University of Edinburgh, a 5 year integrated, outcomes based programme with a spiralling curriculum consisting of distinct modules and vertical themes. The 13 overarching programme outcomes define the attributes of a successful graduate and are used to plan learning opportunities and assessment strategies. The course structure was implemented in 1998, and between 2007 and 2009 there were no major curricular changes.
A questionnaire study was designed and piloted in 2007. FY1s and ES were asked to rate the preparedness of FY1s in a variety of domains that mapped onto the main Edinburgh MBChB programme outcomes, as shown in box 1. In order to keep the survey relatively short, the questions were intentionally broad and subsequent free text areas allowed elaboration and clarification.
The 13 questions relating directly to the programme outcomes of the Edinburgh MBChB, shown with the Foundation doctor stem
Please rate your preparedness for practice as a Foundation doctor at the point of graduation from medical school in the following domains (underlying relates to abbreviations used in figure 1):
Ability to carry out a consultation with a patient (history, examination)
Ability to provide immediate care of medical emergencies, including first aid and resuscitation
Ability to assess clinical presentations, order investigations, make differential diagnoses, and negotiate a management plan
Ability to carry out practical procedures (eg, venepuncture)
Ability to communicate effectively in a medical context
Ability to prescribe drugs
Ability to apply ethical and legal principles in medical practice
Ability to assess psychological aspects of a patient's illness
Ability to apply the principles, skills and knowledge of evidence based medicine (EBM)
Ability to use information and information technology (IT) effectively in a medical context
Ability to apply scientific principles, method and knowledge to medical practice and research
Ability to work effectively in a healthcare system and engage with population health issues such as social aspects of a patient's illness and health promotion
Ability to adopt a self directed and reflective approach to own clinical practice, ongoing learning and professional development.
Questions linked to the predefined domains shown in box 1 were used to address aims 1 to 3. Questionnaire responses were scored using a four item Likert scale (poor, satisfactory, good, and very good). In order to address aim 4, two areas for free text answers were provided following the statements:
Please provide comments to clarify any of your answers to the above questions.
Are there any other specific points you wish to bring to the attention of the medical school in relation to undergraduate medical education in Edinburgh?
Around half of all Edinburgh graduates remain in South East (SE) Scotland to undertake Foundation training. The questionnaire was sent electronically in February 2008 via the postgraduate institute to all FY1s who had graduated from the University of Edinburgh in 2007 and were working within SE Scotland (53% of the total graduate cohort). Graduates of other medical schools working in SE Scotland were not surveyed. Responses were completed online and returned electronically using a web based questionnaire tool. Final datasets were downloaded anonymously and no attempts to identify individuals were made. On receipt of the questionnaire, graduates would have undertaken almost 6 months of work as an FY1 and thus would have gained reasonable insight into the demands and expectations of the role. The same questionnaire with a different introductory sentence was simultaneously sent to all FY1 ES in SE Scotland who supervised 2007 Edinburgh graduates. General reminders were sent electronically approximately 4 weeks and 8 weeks after distribution of the questionnaire. For the subsequent 2 years, the questionnaire was repeated in identical format to obtain information relating to 2008 and 2009 graduates. Around half (49%) of the 125 ES who were surveyed over the 3 year period only supervised Edinburgh graduates for 1 of the 3 years. Ethical approval for the study was waived by the SE Scotland Research Ethics Service.
Questionnaire responses were scored as follows: poor=1, satisfactory=2, good=3, and very good=4. In order to address aims 1 and 2, simple descriptive statistics (mean, standard deviation (SD) and 95% confidence intervals (CIs)) were calculated separately for all FY1 and ES scores in each domain over each of the 3 years. Using combined data from all 3 years, differences between FY1 and ES mean scores in each of the 13 domains were analysed using the unpaired t-test to address aim 3. A value of p<0.004 was considered statistically significant for the purposes of this study (5% significance level with Bonferroni correction for 13 comparisons). Statistical analysis was performed using Microsoft Excel 2003.
In order to elicit additional areas considered important in preparing for practice (aim 4), free text responses were analysed thematically. Open coding was undertaken using NVivo8 software which allows development of a cross-group thematic framework while retaining the ability to check contextual validity and source (eg, year) of individual comments. Two researchers independently assigned codes to emerging areas of interest, constantly renaming, redefining, and reorganising the codes to build a thematic grid.10 Following initial coding, the two researchers discussed differences in the emergent themes until agreement was reached. The data were then recoded by both researchers and the cross-check was repeated, with persisting differences again discussed to agreement.
Total response numbers and rates for each cohort are summarised in table 1. There are a smaller number of ES than FY1s in each year cohort as some consultants provide educational supervision to several Foundation doctors. Overall response totals across the 3 years were 107 FY1s and 85 ES.
Perceived preparedness in predefined domains
For the FY1 and ES datasets, the mean (SD) score for each domain within each year group is shown in table 2. In addition, the table shows the combined means for each domain using the data from all 3 years. For ease of analysis, poorer mean scores are represented by darker shades. The shading highlights the consistency in the scores that were obtained within each group across the 3 years. Table 2 shows that across all 3 years, FY1s felt most prepared in their ‘ability to carry out a consultation’ and least prepared in their ‘ability to prescribe drugs’. ES considered FY1s to be most prepared in relation to their ‘ability to use information and information technology’ and least prepared in their ‘ability to carry out practical procedures’. The Likert scale used in the questionnaire employed a rating of 2 to indicate ‘satisfactory’ preparation for starting work as a doctor within a particular domain. None of the mean FY1 scores fell below a value of 2 across the 3 years studied, indicating that, on average, they felt at least satisfactorily prepared in all domains. Only one of the mean ES scores fell below a value of 2 (1.86 for ‘ability to provide immediate care of medical emergencies’ in relation to 2008 graduates), indicating that this is the only domain in which ES would, in general terms, rate graduate preparedness as ‘unsatisfactory’.
Comparison of perceptions in predefined domains
Using the combined means from all 3 years shown in table 2, two of the top three domains as scored by the FY1s also fall within the top three domains as scored by their ES (carrying out a consultation and communication). Agreement in domains with poorer scores is less consistent, but three of the five lowest scoring domains using combined FY1 scores also fall within the lowest five domains as scored by ES (prescribing, emergency care, and application of scientific method). However, table 2 also shows some striking disparities between the perceptions of the two groups. Using the combined means, the FY1s placed ‘ability to carry out practical procedures’ seventh in the table, but the ES rated them far less prepared in that domain, giving the lowest combined mean. To aid comparison of perceptions, figure 1 is a graphical representation of overall means and 95% CIs over 3 years for FY1s and ES. The p-values from the unpaired t-test are displayed above each pair; the differences between nine of the 13 pairs are statistically significant.
Additional areas of importance
Over the 3 years, a total of 156 comments were made in response to the two free text questions, 47 from FY1s and 109 from ES. Seventy-seven of the comments related to the first free text question and 79 to the second, but given the similarity of content, all free text responses were thematically analysed together. Four major themes arose from the comments as described below.
Theme 1: Knowledge
Despite a specific question enquiring about preparedness to ‘prescribe drugs’, there was discontent expressed by both FY1s and ES in relation to pharmacology knowledge and practical prescribing ability.
FY1 (2008): “I feel that my knowledge of pharmacology was poor compared to other aspects of medicine…”
ES (2007): “I would regard most FY1s as frankly reckless in their prescribing…”
Anatomy and physiology were other areas in which respondents felt that FY1s lacked knowledge, particularly in comparison with their predecessors. Additionally, FY1s were felt to have difficulties translating knowledge into practice.
ES (2008): “Theoretical knowledge of anatomy, pathology and clinical subjects is not as strong as it used to be and this now hinders teaching in clinical years.”
ES (2008): “Good theoretical knowledge, but inexperienced at putting this into practice.”
Theme 2: Skills
Identification and management of acutely unwell patients appeared to be a source of concern for both ES and FY1s.
FY1 (2009): “The one set of scenarios in which we do need to act as such is very acute emergencies—an area in which we received far too little training given the responsibilities in this respect which circumstances often place on us.”
ES (2009): “They are not always good at recognising an acutely unwell patient or identifying those that need to be prioritised.”
Respondents placed great emphasis on non-technical skills including decision making, initiative and prioritisation.
FY1 (2008): “I felt under-prepared for making clinical decisions…”
ES (2007): “Very few of the FY1 doctors will take any initiative…”
FY1 (2009): “We received absolutely no training in most of the critical day-to-day-relevant aspects of being an FY1: prioritising, managing and keeping track of a large workload of tasks and jobs, many of which will be completely unfamiliar…”
Interpersonal non-technical skills received more favourable comments from both groups. While the FY1s were generally felt by their supervisors to be effective and sensitive communicators with patients, inter-professional communication, including referrals and ward-round presentations of patients, received more critical comments.
ES (2008): “Although good communicators with patients, they are generally sub-optimal in their communication with other medical staff…”
A series of comments suggested that supervisors are concerned by the level of stress experienced by newly qualified doctors and that the FY1s were not optimally equipped with strategies to cope with stress.
ES (2008): “I have been concerned about the amount of sick leave FY1s take usually related to stress.”
FY1 (2007): “I'm sure it must be normal for graduates to feel out of their depth when starting work, but in retrospect many parts of the curriculum seem poorly designed to help us meet this challenge as well as we might.”
Theme 3: Personal attributes
There were many comments relating to personal attributes of Edinburgh graduates. Problems relating to lack of confidence were mentioned by both ES and FY1s.
ES (2007): “They have a great deal of knowledge but little confidence…”
FY1 (2008): “I … was lacking confidence in putting pen to paper as we went from no responsibility to high levels of responsibility overnight.”
Other largely complimentary comments from ES related to enthusiasm, reliability, and other aspects of professionalism.
ES (2009): “Most FY1s are bright, keen and hard-working and a pleasure to work with.”
Theme 4: Familiarity with ward environment
Both FY1s and ES felt that familiarity with the environment of the wards was an important component of transition from medical student to FY1. Comments from FY1s suggested that spending longer on the wards would result in increased familiarity with the day-to-day jobs involved in ‘running a ward’, incorporating both clinical and administrative duties.
FY1 (2007): “The only aspect of preparation for practice that I felt I lacked when starting was the practical experience of running a ward.”
FY1 (2008): “Undergraduates should have more exposure to everyday ward work and tasks, to better prepare them for FY1.”
FY1s graduating from Edinburgh medical school between 2007 and 2009 felt that their preparation for Foundation training was good in five out of the 13 MBChB programme outcomes, and satisfactory in the remaining eight. Over the same period, FY1 ES felt that the preparation for Foundation training had been satisfactory in all 13 domains. The perceptions of graduates and their ES were significantly different in the majority of domains. However, the additional aspects of preparing for practice that were identified by the two cohorts showed remarkable similarity.
In concordance with other studies, FY1s in this study consistently scored themselves significantly higher than did their ES in the majority of domains.5 6 It is possible that some of this difference may be due to ES bias against the domains themselves, perhaps viewing some of them as irrelevant to everyday clinical practice. The greatest disparity in the perceptions of the two groups related to ‘ability to carry out practical procedures’, echoing the results of a previous study.5 A South African study that correlated newly qualified doctors' self-assessment scores in practical procedures with OSCE (objective structured clinical examinations) scores also demonstrated misplaced confidence.11 The ‘ability to provide immediate care of medical emergencies’ was the only domain in which preparation of any graduate cohort was deemed, on average, to be unsatisfactory. Concerns relating to the care of acutely unwell patients were also evident in the qualitative data. Such findings concord with other studies and lack of preparedness in this domain appears to be a perennial problem, both within the UK and throughout the world.5 12–15
Analysis of the free text responses highlighted a number of areas that had not featured in the questionnaire, yet were felt by respondents to be important components of preparedness for practice. While the questionnaire specifically asked respondents to score ‘ability to communicate effectively in a medical context’, it did not differentiate between communication with patients and colleagues. These two types of communication present subtly different challenges for newly qualified doctors, and preparedness for each is important. The free text comments indicated that some FY1s and their ES felt that they had been well prepared to communicate with patients and relatives, but less prepared in relation to communication with colleagues. It is of interest that most other studies in this area do not differentiate between communication contexts.6 8 16 The single study that does make this differentiation concords with our findings: consultants and specialist registrars felt that FY1s were better prepared for communicating with patients and relatives than with medical colleagues.17
Opinions regarding non-technical skills such as decision-making, initiative and prioritisation were not specifically sought within the questionnaire, but were attributed importance by both FY1s and their ES. Task prioritisation has previously been identified as an important component of the FY1 role which is usually learnt ‘on the job’, making doctors in their early days feel unprepared.13 18 The first postgraduate year is renowned for being a stressful and difficult year.19 At least some of the stress experienced by newly qualified doctors seems to relate to exposure to specific events such as acutely unwell patients, on-calls or night shifts.13 It may also relate to commencing a new placement with insufficient induction processes and uncertainty of role.20 Suggestions to improve preparedness for practice made by respondents to our questionnaire included encouraging students to spend longer on the wards to increase familiarity with day-to-day jobs and increased shadowing time. These suggestions echo the findings of a large study commissioned by the General Medical Council13 that informed the 2009 version of Tomorrow's Doctors1 recommendation of a Student Assistantship period (in which students take on the role of Foundation doctor) as an integrated part of primary medical training. It will be interesting to track preparedness for practice in future years, both in our institution and UK wide, as these new standards are implemented.
This study combines the strengths of three consecutive years' data with two different perspectives. It is, however, limited by its poor response rate and narrow setting within Edinburgh medical school. Furthermore, only Edinburgh graduates who took up FY1 posts within SE Scotland were surveyed. It is possible that those who responded to the survey either had particularly strong feelings on the preparedness of Edinburgh graduates to begin clinical practice or, in the case of FY1s, felt prepared enough to devote time and energy to an optional questionnaire. The responses may therefore not be representative of the whole cohort, and consideration needs to be given to incentives and other methods of improving response rates in future years. New national application procedures for Foundation training have resulted in a gradual decrease in the number of Edinburgh graduates remaining in SE Scotland. It is therefore essential that the medical school finds ways of maintaining contact with graduates who have moved further afield and even outwith the UK. The use of a questionnaire as a data collection method limited the study in terms of discovering why respondents hold the views that they do, a question best answered using interview based techniques. In addition, the study sought information on perceived preparedness as opposed to actual preparedness, two variables which cannot be assumed to correlate. A recent Japanese study, for example, found no correlation between pass rate on the National Medical Licensure Examination and perceived preparedness for practice in any domain.21
When considered in the context of previous research, this study has identified several areas requiring further work. The suggestion that FY1s are better prepared to communicate with patients and relatives than with colleagues has received little attention in the literature and warrants further exploration. A more detailed understanding of the specific challenges faced by newly qualified doctors when managing an acutely unwell patient is required, so that this perennial problem may be tackled. In addition, improving the abilities of newly qualified doctors to self-assess their competence, particularly in relation to procedural skills, is crucial to ensuring the safety of patients.
In evaluating preparedness for practice, valuable insights may be gained by looking beyond core curricular competencies to include other skills that are required by newly qualified doctors such as decision-making, initiative, prioritisation, and coping with stress. In addition, medical schools may glean more valuable information by differentiating between patients and colleagues in questions asking about communication skills. Education for Foundation trainees could focus on the areas in which newly qualified doctors are perceived to be less prepared, such as managing acutely unwell patients, prescribing, and procedural skills.
Preparedness for practice data may be enriched by repeated collection over several years and incorporation of free text responses.
The non-technical skills of decision-making, initiative, prioritisation, and coping with stress may be important components of preparing new graduates for practice.
Curriculum development, in particular the UK Student Assistantship, should emphasise the specific challenges of acute care and prescribing.
Education for Foundation trainees could focus on the areas in which graduates are perceived to be least prepared, such as acute care, prescribing, and procedural skills.
Current research questions
Why do newly qualified doctors feel better prepared for communication with patients than colleagues?
What are the specific challenges faced by newly qualified doctors when caring for acutely unwell patients?
How can the abilities of medical students and new graduates to self-assess their competence in procedural skills be improved?
With thanks to all staff and students who completed the questionnaires, to Ms Marion Mackay, Ms Margo Armit and Ms Katie Morgan for administrative support, and to Professor Jim Byrne, Dr Katharine Byrne and Ms Cat Graham for their advice on the statistical analysis.
Funding Clinical Skills Managed Educational Network c/o Clinical Skills Centre Ninewells Hospital, Dundee, UK.
Competing interests None.
Ethics approval Ethical approval was waived by the South East Scotland Research Ethics Service.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.