Article Text
Abstract
Purpose The Good Intern Programme (GIP) in Sri Lanka has been implemented to bridge the ’theory to practice gap’ of doctors preparing for their internship. This paper evaluates the impact of a 2-day peer-delivered Acute Care Skills Training (ACST) course as part of the GIP.
Study design The ACST course was developed by an interprofessional faculty, including newly graduated doctors awaiting internship (pre-intern), focusing on the recognition and management of common medical and surgical emergencies. Course delivery was entirely by pre-intern doctors to their peers. Knowledge was evaluated by a pre- and post-course multiple choice test. Participants’ confidence (post-course) and 12 acute care skills (pre- and post-course) were assessed using Likert scale-based questions. A subset of participants provided feedback on the peer learning experience.
Results Seventeen courses were delivered by a faculty consisting of eight peer trainers over 4 months, training 320 participants. The mean (SD) multiple choice questionnaire score was 71.03 (13.19) pre-course compared with 77.98 (7.7) post-course (p<0.05). Increased overall confidence in managing ward emergencies was reported by 97.2% (n=283) of respondents. Participants rated their post-course skills to be significantly higher (p<0.05) than pre-course in all 12 assessed skills. Extended feedback on the peer learning experience was overwhelmingly positive and 96.5% would recommend the course to a colleague.
Conclusions A peer-delivered ACST course was extremely well received and can improve newly qualified medical graduates’ knowledge, skills and confidence in managing medical and surgical emergencies. This peer-based model may have utility beyond pre-interns and beyond Sri Lanka.
- patient safety
- education and learning
- acute care
- clinical skills
- peer learning
Statistics from Altmetric.com
Introduction
Newly qualified doctors undertaking internships often carry a large responsibility in patient care, yet with little practical experience, which is associated with adverse consequences.1 2 The introduction of newly qualified doctors into the clinical team is associated with omissions in the recognition of deteriorating patients and failed communications to senior team members, which can result in a failure to escalate treatments.3 The ability to respond effectively to such emergency scenarios requires clinicians to be confident in applying their knowledge and clinical skills in practice.3–5 It is often the junior members of the clinical team led by junior doctors who are the point of first contact for acutely unwell patients. Whereas undergraduate medical training lays the foundation for the knowledge and analytical skills that are required by junior doctors, there remains a gap in the translation of this knowledge into the skills needed to act quickly and confidently in emergency situations.4 6
In high-income countries including the UK, clinical and decision-making skills training has been increasingly incorporated into final year medical undergraduate training programmes to assist students in their transition to practice, and peer learning has been explored as a method of delivering skills training for junior doctors and nurses.7–9 There is evidence that such a model may offer effective knowledge transfer, improve the ability of junior clinicians to problem solve and make decisions based on systematic assessment in emergency scenarios.10 11 In low and middle-income countries (LMIC), practical acute care skills training for front-line staff is limited, despite ongoing remedial efforts.9 11 Unaffordability of training courses, unavailability of suitable trainers, traditional or didactic models of learning and lack of course material and equipment are contributory factors. In Sri Lanka, a LMIC, 1000 medical graduates begin internship annually, at least 150 of whom are overseas qualified graduates with limited exposure to the local healthcare system.12
Historically, there has been minimal opportunity for these students to prepare for the practical components of internship and no opportunity for structured handover between the outgoing junior doctors and the incoming newly qualified doctors. As such, there is little capacity for colleagues to share experiences or knowledge of the structures and processes necessary to survive life on the front line. On publication of their final results, graduates have an approximately 9-month gap (an administrative period) before commencing their first clinical post. During this time, many are involved in administrative or support roles within the university, with little opportunity to develop or consolidate clinical skills.
Since 2014 the ‘Good Intern Programme’ (GIP)13 has been piloted in Sri Lanka to help prepare graduates for transition to practice. This multi-component programme for newly qualified doctors includes essential information on healthcare system infrastructure, cognitive skills for prescribing and medical documentation alongside lessons in regional dialects. In this paper we report the evaluation of a peer-delivered (fellow graduates from the same intake) practical 2-day acute care skills training (ACST) course designed and delivered by a peer faculty, to improve participants’ knowledge, confidence and self-assessed skills for managing common acute medical and surgical emergencies.
Study design
Needs analysis
All newly qualified doctors awaiting internship were contacted via their respective universities and their trainee representatives and invited to participate in an online anonymous needs analysis to rate their competence and confidence to perform essential clinical skills. The survey included self-evaluation of graduates’ ability to undertake structured assessment, manage acutely unwell patients and perform practical skills synonymous with advanced life support, including defibrillation, a structured systematic (A–E) assessment and airway manoeuvres. The findings of the needs analysis were used to devise the ACST course.
Course design and delivery
The course content was developed iteratively by an interprofessional collaboration including senior intensive care medicine trainee and nurses from the UK, senior physician and education fellow practising in Sri Lanka, international clinical educators with expertise in critical care, resuscitation and simulation, and junior doctors who had recently completed an internship in Sri Lanka. The course was designed to be practical, using skills stations, clinical scenarios and small group sessions to maximise participant participation and build confidence. Clinical scenarios were selected using a Delphi process,14 reflecting situations commonly encountered by junior doctors working in acute medical hospitals in Sri Lanka and for their opportunity to integrate essential management guidelines. Scenarios included respiratory failure, sepsis, acute coronary syndrome, anaphylaxis after anti-venom, dengue fever and organophosphate poisoning.
The peer faculty was selected from newly qualified doctors participating in a series of Train the Trainer (TTT) sessions based on established methodology.7 9 10 15 Participants showing aptitude for teaching clinical skills set and interest in building capacity within their profession were given preference to join the faculty.8 9 The faculty received training in skills instruction, facilitation, assessment (OSCE) structure and good practice, session setup, giving feedback and marking prior to the course, and undertook observed practice.
The 2-day course was conducted for small groups of up to 20 doctors over 4 months. Enrolment in the programme was voluntary, with publicity aimed at all newly qualified doctors awaiting internship who had responded to the needs analysis. Synonymous with undergraduate training, English was the medium of instruction with discussion sometimes continuing in the local languages. Participants were invited to access course materials online (lecture materials, guidelines, audio lectures) using a learning management system14
Course assessment and feedback
Graduates’ knowledge was assessed by a randomised pre- and post-course multiple choice test of 20 questions (MCQ) selected from a question bank designed to minimise sharing across participant groups. Participants were invited to assess their confidence after each of five skills stations (figure 1) and at the end of the course using a 5-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’.16 Skills ability (skills listed in figure 2) was rated by participants pre- and post-course using a 10-point Likert scale self-assessment questionnaire, comparable to those completed by doctors during foundation training in the UK.16–18
Session by session feedback from skills stations.
Participants’ pre- and post-course self-assessment of skills.
In addition, five Objective Structured Clinical Examination (OSCE) stations were used to provide participants with objective feedback following course completion. The five stations covered were safe defibrillation, recognition and management of tachyarrhythmia, basic life support (BLS), A–E structured assessment and emergency management of the airway.19 A structured marking criteria based on participants’ ability to perform each aspect of the skill safely and competently was used.20 The pass mark was set at 50%. In addition to the TTT sessions outlined above, the faculty undertook paired assessment for each OSCE station to ensure validity and consistency in marking across the faculty group.9
Participants were invited to provide feedback rating their learning experience, the opportunity to actively participate in the skills stations and scenarios, course delivery and whether they felt learning objectives were achieved using the 5-point Likert scale described above. This feedback was collated electronically following each session and on completion of the course. Suggestions for course improvement were requested in free-text format. A further five questions (figure 3) were used during feedback for the last three courses to evaluate the peer teaching process. Participants were not approached for these questions in order to limit the amount of feedback being sought from candidates. Course participation was voluntary with assessments and feedback also voluntary and anonymised. All candidates had the option to opt out of their information being used for analysis and research. No further ethical review was sought.9
Session by session feedback from scenarios.
Data analysis
Data were analysed using Stata Corp 13.21 Summaries of the results for the MCQ and OSCE were presented as mean (SD) for continuous variables and as counts and percentages to describe discrete variables. The Wilcoxon signed-rank test was used to analysis paired non-parametric continuous variables and paired Likert scale variables. All tests were two-sided and the level of significance was taken as 0.05. The free-text suggestions for improvement provided by the participants in the feedback were listed by two investigators and coded for analysis.
Results
Needs analysis
Seven hundred and thirty-two of 902 (81.1%) newly qualified doctors who responded to the survey wished to participate in a 2-day practically focused ACST course. Eight hundred and seventy-five (97%) responders stated they would benefit from such an opportunity as part of preparation for internship. Five hundred and eighty-six of these 902 (64.9%) doctors reported that they had never practised advanced airway manoeuvres or defibrillation either in vivo or in a training environment. Six hundred and six of the 902 participants (67.1%) surveyed indicated that they were not confident in performing these skills without supervision. Furthermore, 433 of the 902 (48.1%) reported lack of confidence in interpreting key investigations for management of emergency situations, including ECG and arterial blood gas (ABG) results.
Course delivery
Between July and October 2015, 17 courses were delivered by a faculty consisting of eight peer trainers, training 320 newly qualified doctors at venues across Colombo.22 The estimated cost of delivering this 2-day course including refreshments in Sri Lanka was approximately 30 British pounds (GBP).
Course assessment and feedback
The mean (SD) score of the MCQ pre-course was 71.03 (13.19) compared with 77.98 (7.70) post-course (p<0.05, n=320). Of the 283 participants who provided feedback, 275 participants (97.2%) reported that the course had increased their overall confidence in managing ward emergencies (figure 4). Participant feedback describes an increase in confidence in performing defibrillation (98.9%, n=280), BLS (98.3%, n=278), A–E assessment (96.8%, n=274) and ABG interpretation (99.2%, n=281) (figure 1). Participants reported significantly higher (p<0.05) self-assessment ratings in all 12 assessed skills post-course compared with pre-course (figure 2).
Overall feedback from course participants.
All course participants chose to participate in the five station OSCE. The mean (SD) score for tachyarrhythmia management OSCE was 56.77 (29.60) and for airway and breathing OSCE was 54.98 (34.03). The lowest mean (SD) score was observed for A–E assessment (49.94 (29.15)) while the highest score was observed for BLS (67.02 (36.86)) and safe defibrillation (57.65 (33.95)), with 232 (72.50%) and 206 (61.39%) of participants scoring over 50%, respectively.
Of the 320 newly qualified doctors who participated, 283 provided feedback on the overall course. All respondents reported that the course improved their knowledge and 282 participants (99.3%) reported an improvement in their ability to perform skills (figure 4). In feedback following the clinical scenarios, participants reported that the peer faculty had provided an effective learning environment and given constructive feedback for these scenarios (figure 5). A subsample of 66 participants from the last three courses reported that peer learning was an effective method of knowledge sharing, learning and confidence building (figure 3).
Participant feedback regarding peer-to-peer learning method.
A total of 103 of the 320 participants (32.25%) gave recommendations for improvement. Of these, 42 (41.2%) recommended that more medical and surgical scenarios would be beneficial for their clinical practice and 50 (48.3%) suggested that the addition of obstetric and paediatric emergency scenarios would be of benefit.
Discussion
The needs analysis highlighted the limited opportunities available during undergraduate training to practise essential clinical skills required for managing emergency clinical situations. The ACST course conducted for 320 graduate participants preparing for internship in Sri Lanka demonstrates that this co-designed peer-delivered course significantly increased the knowledge, confidence and skills necessary to manage acute medical and surgical emergency scenarios of graduate doctors preparing for their clinical practice. The OSCEs indicated that the majority of participants could demonstrate effective application of clinical knowledge within realistic clinical scenarios.
The post-course MCQ scores were significantly higher than the pre-course MCQ scores. Greatest increase in both skill and confidence was reported in practical resuscitation skills such as defibrillation, BLS and ABG interpretation and when all candidates were enabled to participate and ask questions. This is perhaps not surprising given the low pre-course confidence levels reported by both the needs assessment and pre-course self-assessment. More advanced skills such as tachyarrhythmia management and intubation also resulted in increased ability, but with a less marked increase in confidence—perhaps reflecting the complex nature of these procedures.
A prioiri knowledge from the expert faculty informs that junior doctors are called on to deliver complex skills such as intubation during internship in Sri Lanka. Achieving a measurable increase in competency of such complex skills in this learning forum is unrealistic; however, it may be argued that inclusion of such skills provides the opportunity for candidates to gain a level of situational awareness and preparedness, which is essential for safe delivery of such interventions that they would not otherwise experience. Internationally, similar programmes have highlighted that, while mastery of such complex skills requires repeated immersion in the situation, scenario-based practical skills training may offer insights and useful preparatory learning for candidates.23 24
The results of the needs analysis, participant self-assessment and feedback further add to the growing body of evidence in support of interactive practical training methods, such as those used during this course, to improve recognition and management of acutely unwell patients.24 25 These practical training methods provide an unique opportunity for learners to gain experience and confidence in complex clinical decision-making needed to respond to emergency scenarios, and to reduce the incidence of failure to rescue without jeopardising the safety of patients or colleagues.4 7–9 11
This ACST course, perhaps uniquely, was designed and delivered by a group of newly qualified doctors. Course feedback demonstrates that participants found their peer faculty to be professional, knowledgeable and competent to deliver the course (figures 2–5). Although requiring investment of time and expert faculty up-front, the TTT model7–9 equipped newly qualified doctors with the skills and capacity to deliver an effective peer learning course and offers a sustainable solution for delivering similar courses with each batch of new graduates.
Similarly, the cost per participant was relatively modest compared with many expert faculty- or simulation-delivered courses.24 While expertise and training techniques including high-fidelity simulation remains an essential part of developing specialist skills for trainees dealing with uncommon and complex emergencies,24 26 these results would suggest that peer-delivered practical courses may be a cost-effective training method for large cohorts of medical graduates in preparation for managing common medical and surgical emergencies in clinical practice.27 This co-designed peer delivery method may also be beneficial for more senior doctors and interdisciplinary teams working in specialist areas, including critical care, to help refresh knowledge and provide an opportunity for sharing both knowledge and skills.4 24 27
Limitations
Only 320 of the 902 needs assessment respondents attended the ACST course. This may in part have been due to the course venues only being available in Colombo. Participant numbers were limited by limited time available before commencement of internship in this inaugural year of the ACST course and high faculty to participant ratio. This led to some potential registrants not being invited to attend. An earlier start to the course in coming years and a larger group of peer trainers may help overcome this limitation. Newly qualified doctors delivering the course need repeated TTT and upskilling each year. It is anticipated that, over time, experienced faculty will return to facilitate junior faculty upskilling. Long-term knowledge retention and effect on patient outcomes are unknown and warrant further exploration. Paired (pre- and post-course) self-assessment was used as the primary tool to evaluate skills gain. The validity of this method—although common in both undergraduate and postgraduate medical training—is questioned. In postgraduate training, doctors are often thought to overestimate their ability when completing self-assessment.17 18 28 Perspectives of including more complex skills in such a course are discussed above. Course delivery and OSCEs were facilitated by the same faculty, resulting in the potential for scoring bias. This methodology, while clearly having limitations, reflects the limited availability of skilled resources in this setting and mirrors practices of similar programmes internationally.9 29 30
Conclusion
This paper demonstrates the feasibility of a peer-delivered training course in improving the knowledge, confidence and skills of newly graduated doctors in common medical and surgical emergencies as they prepare for clinical practice. Similar peer-based courses may have utility beyond newly qualified doctors and beyond Sri Lanka for honing essential practical skills in front-line staff in other LMIC settings.
Main messages
Peer training provides a sustainable method for delivering a practical skills programme to bridge the theory to practice gap for newly qualified doctors.
A clinically-focused practical skills programme builds perceived confidence for newly qualified doctors as they prepare for internship.
Peer-to-peer training enables continued professional development for doctors working in low and middle income countries where there might otherwise be limited opportunity for such learning.
Current research questions
Is a peer-delivered practical skills programme to improve confidence of newly qualified doctors in the management of common medical and surgical emergencies transferable to other LMICs?
Does a peer-delivered practical skills programme improve clinicians' performance in response to common acute medical and surgical emergencies?
What is the impact of a peer-delivered practical emergency care skills programme for junior doctors on confidence in clinical practice: a 1-year follow-up evaluation?
Acknowledgments
The ACST course was a collaboration between the Government Medical Officers Association (GMOA, the doctors' trade union), the Ministry of Health (Education, Training and Research directorate) and the Network for Improving Critical care skills and Training (NICST). NICST has been delivering short courses for doctors and nurses, focused on improving the care of the acute and critically unwell, since 2014. The programme was delivered free as the GIP in 2015 was funded by the Ministry of Health (MOH). Collaboration has enabled this course to be integrated into a wider programme of preparing medical graduates for clinical practice from 2016 onwards. The MOH, GMOA and NICST are currently coordinating the 2016/17 ACST as part of GIP. Such a collaboration has an important role in achieving sustainability of such a course.
References
Footnotes
Contributors Study design was done by AB, AP, PDS, TS, SDA, PGM, PCS, PW, DR, EMD, TW, KT
and RH. Data collection was carried out by EMD and TW. Overall technical data analysis.
and review were done by AB, AP, PDS, TS, SM, SJ, AD and RH. All authors provided.
significant contributions to the writing and revision of the manuscript.
Funding The courses were part funded by Education, Training and Research unit, Ministry of Health, Sri Lanka.
Provenance and peer review Not commissioned; externally peer reviewed.