Background The COVID-19 outbreak has dramatically impacted medical education, both bedside and academic teaching had to be adapted to comply with the reorganisation of care and social distancing measures.
Objectives To overview the impact of the pandemic on medical education, including the pedagogical responses adopted and their assessment by medical students and residents.
Material and methods This restricted systematic review was performed using Rayyan QCRI, to select observational or interventional articles and field experience reports assessing the impact of the COVID-19 pandemic on medical education for medical students and residents. Study design, study population, geographical origin, use of an educational tools (including softwares and social media), their type and assessment, were recorded. For studies evaluating a specific tool the Medical Education Research Study Quality Instrument (MERSQI) was used to assess study quality.
Results The literature search identified 1480 references and 60 articles were selected. Most articles focused on residents (41/60; 69%), and half (30/60; 50%) involved surgical specialties. Online courses were the most frequently used pedagogical tool (52/60; 88%). Simulation tools were used more frequently in articles involving surgical specialties (15/29; 52%) compared with medical specialties (2/14; 12%) (p=0.01). Only four studies reported the assessment of pedagogical tools by medical students, their MERSQI scores ranged from 5.5/18 to 9.0/18.
Conclusion Medical education was highly impacted by the COVID-19 pandemic particularly in surgical specialties. Online courses were the most frequently attempted solution to cope with social distancing constraints. Medical students’ assessment of pedagogical tools was mostly positive, but the methodological quality of those studies was limited.
- medical education & training
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This article is made freely available for use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage
Statistics from Altmetric.com
Since March 2020, the world has been facing the COVID-19 pandemic and despite the development of several vaccines,1 the situation remains critical and the pandemic uncontrolled. Not only have healthcare systems been dramatically impacted but most governments have also adopted nationwide emergency measures, including closure of universities and lockdowns to contain the spread of the virus. The COVID-19 pandemic has disrupted medical education revealing its strengths and weaknesses. As a consequence, on the one hand in most countries medical students were excluded from in-hospital daily activities, in-persons classes and clinical rotation2; on the other hand, medical residents were involved in the management of COVID-19 patients, and non-urgent staff, meetings, conferences, in-persons classes, elective surgical procedures and clinical rotations were cancelled.3 Thus, all educative programmes and internships had to evolve from in-person to remote learning using various tools.4 This unprecedented pandemic has provided an opportunity to take stock of the resources available, to highlight the shortcomings, to test numerous innovations in the field of digital learning and simulation, and perhaps to implement lasting changes in the teaching of medical students and residents, in the faculty as well as at the bedside.
The aim of this restricted systematic review was first, to report an overview of the impact of the COVID-19 pandemic on medical education; second, to report which pedagogical solutions were tried; and lastly to report medical students' and residents’ feedback.
Material and methods
This restricted systematic review has been performed according to the flexible framework for restricted systematic reviews published by the Centre for Evidence-Based Medicine, University of Oxford5 and to the Synthesis without meta-analysis guideline.6
Literature search and information sources
We searched MEDLINE/PubMed (Education Resources Information Centre, a specialised search education database) and the Cochrane Database of Systematic Reviews until 1 June 2020 for original articles and reviews restricted to French and English Language. The search strategy combined free text search, exploded Medical Subject Headings (MeSH) terms. The grey literature was not explored.
The PubMed search equation was the following: ((medicine/education[Mesh] OR general surgery/education[Mesh] OR surgery/education[Mesh]) OR medical education[MeSH Terms]) OR (continuing medical education[MeSH Terms]) OR (medical students[MeSH Terms]) OR (academic training [MeSH Terms]) OR (medical education) OR (continuing medical education) OR (medical students) OR (faculty practice) OR (academic training)) AND ((coronavirus OR “corona virus” OR coronaviridae OR coronaviridae OR betacoronavirus OR covid19 OR “covid19 19” OR nCoV OR “CoV 2” OR CoV2 OR sarscov2 OR 2019nCoV OR “novel CoV” OR “Coronavirus” [Mesh] OR “Coronavirus Infections” [Mesh] OR “covid19” [Supplementary Concept])).
Study selection and eligibility criteria
Observational or interventional articles and reviews were considered if (1) they assessed the impact of the COVID-19 pandemic on medical education, (2) the study population was medical students and residents. A single reviewer (FC) screened titles and abstracts after removing duplicates. After reading full text of preselected manuscripts, three types of articles were included by two investigators (MB and AL):
Field experience reports describing pedagogical tools used or changes made during the COVID-19 pandemic in order to maintain medical education in a specific setting (a country, a teaching hospital, a medical or surgical specialty…).
Observational and interventional articles reporting either the development of a new pedagogical tool during COVID-19 pandemic and its assessment by medical students or residents.
Surveys measuring the impact of the COVID-19 pandemic on medical education.
Articles were excluded if:
The period did not correspond to the COVID-19 pandemic.
The involved students were not medical students or residents.
They were editorials or letters with ineligible outcomes.
They dealt with the impact of the COVID-19 on students’ evaluation: exams/applications.
The topic was the deployment of students to manage COVID-19 patients.
The language was not English or French.
Data extraction and analysis
The recorded information for each selected study included the study design, study population (medical student vs resident), country of the study, involved specialty (medical vs surgical and type). The changes made as well as the list of educational tools used to preserve medical education during the COVID-19 pandemic were assessed. Educational tools were classified as follows:
Online courses which were subdivided in 10 subgroups: lectures, tutorials, podcasts, webinars, journal club, virtual conferences, virtual cases reviews, web-based video, morbidity and mortality review and written material.
e-learning programmes defined as structured educational programmes using electronic and/or interactive tools.
Telemedicine defined as the use of telecommunications technology to maintain interaction between students and patients (virtual visits, teleconsultation…).
Virtual educational tool/simulation: defined as the use of simulation or virtual augmented reality tools.
Software and social media used were also recorded. Data were extracted independently by two investigators (FC and either MB or AL). Disagreements were discussed and resolved by consensus between the investigators. Because of the heterogeneity of medical education systems and included articles, we did not conduct a meta‐analysis. Results are presented as narrative synthesis with summary tables and figures.
Quality of included articles
For quantitative studies that evaluated a specific tool for medical education during the COVID-19 pandemic, we used the Medical Education Research Study Quality Instrument (MERSQI) to assess study quality on 10 criteria: study design; number of institutions; response rate; type of data; internal structure; content validity; criterion validity; appropriateness of data analyses; sophistication of data analyses and outcome level.8 The possible total MERSQI score can range from 6 to 18. Evidence for the validity of the MERSQI has been shown to be associated with acceptance vs rejection of medical education manuscripts.9
Data are presented as median (range) or counts (percentage). We used the Fisher’s exact test to compare qualitative variables. A two-tailed p<0.05 was considered statistically significant. Analyses were performed with JMP software (V.14 (SAS Institute)).
Literature search and characteristics of the included articles
Literature search identified 1480 citations of interest, of which 60 were included in this restricted systematic review3 10–69 (figure 1). Among them, 48 (48/60; 80%) were field experience reports describing pedagogical tools used during the COVID-19 pandemic and 12 (12/60; 20%) were observational or interventional studies assessing a pedagogical tool or using a survey to characterise the impact of COVID-19 on medical education. In the 12 observational/interventional studies, the population ranged from 6 to 852 (not available, NA=1).
Most of articles originated from America: USA (38/60; 63%) and Canada (4/60; 7%). Seven articles originated from Asia (7/60; 12%) and eight from Europe (8/60; 13%) (table 1 and online supplemental table 1).
Most articles focused on residents (41/60; 69%), 11 articles (11/60; 18%) on both residents and medical students and 8 articles (8/60; 13%) on medical students only. Half of articles (30/60; 50%) involved surgical specialties, 16 articles involved medical specialties (16/60; 26%) and 7 (7/60; 12%) articles involved medico-technical specialties (NA 7/60; 12%). The list of specialties is reported in online supplemental table 2.
Educational tools, software and social media used as solutions during the COVID-19 pandemic
The types of educational tools used to maintain medical education during the COVID-19 pandemic among 59 studies with available data are reported in figure 2A. Online courses were the most frequently reported pedagogical tool (52/60; 88%). Among online courses, lectures were found in 34 articles (34/60; 58%), virtual case review in 22 articles (22/60; 37%) and tutorials in 17 articles (17/60; 29%). Furthermore, virtual educational or simulation tools were mentioned in 18 articles (18/60; 30%), an exhaustive e-learning programme was reported in seven articles (7/60; 12%) and the use of telemedicine was reported in 10 articles (10/60; 17%).
The details of software and social media used were available for 33 articles. The use of Zoom (Zoom Video Communications) was reported in 20 articles (20/33; 60%), Google Meet (formerly known as Hangouts Meet) in 6 articles (6/33; 18%) and Cisco Webex Teams and Meeting in 5 articles (5/33; 15%) (figure 2B). Social media were used in 10 articles (YouTube, n=6 (18%) articles, Twitter n=2 (6%) and Facebook n=2 (6%)).
Comparison of pedagogical tools reported between surgical and medical specialties
The comparisons of pedagogical tools used between medical specialties and surgical specialties are reported in table 2. Virtual and simulation tools were used more frequently in articles involving surgical specialties (15/29; 52%) compared with medical specialties (2/16; 12%), (p=0.01). The use of online courses, e-learning programmes and telemedicine was similarly reported by medical and surgical specialties.
Consequences of the COVID-19 pandemic on medical education and assessment of new pedagogical tools by medical students or residents
Eight studies (8/60; 13%) reported the impact of the COVID-19 pandemic on the education of medical students or residents in various surgical specialties using a survey.19 27 32 34 40 51 52 63 64 The main results of these studies are reported in table 3. The evaluated criteria varied across these studies. Most students reported a decrease in patient-contact time, elective surgery activity and indefinite postponement of clinical rotations,19 27 with a negative impact on surgical training and surgical skills acquisition.19 27 32 51 52
As a consequence, most students expressed their concerns regarding career planning and board examinations scores, and thus postponed their examinations.27 32 More than half of students perceived an increase in stress levels,34 an impact on mental health.34 52 Students were more likely to take 1 year off from medical school after than before the start of the COVID-19 pandemic.32
Between 14.8% and 70% of residents were deployed on COVID-19 duty depending the level of COVID-19 incidence in the country.19 27 52 More than half of students had a session about personal protective equipment,34 but between 62% and 97% of residents expressed concerns regarding exposure and personal protective equipment availability.
Most of these studies reported an increase in videoconferencing and remote clinical work.19 27 32 51 52
Only four studies41 46 61 reported the assessment of specific pedagogical tools by medical students (4/60; 7%) during the COVID-19 pandemic using a questionnaire. The MERSQI scores ranged from 5.5/18 to 9/18 (online supplemental table 3). All studies were single group post-intervention descriptive studies only reporting the satisfaction of the students. The main results of these studies are reported in table 4. Three studies investigated remote lectures and virtual cases41 61 and one study investigated remote standardised patient encounters.46 The level of satisfaction was very high for all studies, with an improvement of the relationship with the teacher.41 61
The 48 field experience reports describing tools used or suggested during COVID-19 pandemic are summarised in online supplemental table 4.
This restricted systematic review synthesises the impact of the COVID-19 on medical education and portrays educational solutions attempted for maintaining medical education despite social distancing. About two-thirds of articles focused on residents and 50% referred to surgical specialties. Online courses were the most frequently reported pedagogical tool (52/60; 88%). Virtual reality and simulation tools were reported significantly more frequently in articles involving surgical specialties than in articles involving medical specialties which highlighted that the needs and/or pedagogical interests for medical education are different between medical and surgical specialties. Impact of the pandemic on medical education varies across specialties and depends on the incidence of COVID-19 in the location of the medical education programme.19
Predominance of surgical articles
Most of the articles related to the impact of the COVID-19 pandemic concerned surgical specialties (50%, compared with 27% for medical specialties and 12% for medico-technical specialties), suggesting a higher impact on surgical specialties than other specialties. Indeed, cancellation of all elective surgical procedures has led to a drastic decrease of surgical training and to surgical skill decay. Interestingly, virtual simulation was more frequently reported in surgical specialties than others. This result is consistent with the concerns of residency programmes regarding continuation of surgical skills training. The disruption of surgical training has also led to a mental health impact of surgical trainees leading to questioning the pursuit of a surgical career.27 Conversely, most medical specialty residents have been charged with treating COVID-19 patients.19 This clinical activity was an occasion to train clinical skills such as interviewing, clinical reasoning, supporting patient emotion, counselling or explaining diagnostic test results. However, specialties directly impacted by the management of COVID-19 patients, such as pneumology and intensive care medicine, were under-represented in the literature. Still, most of these specialties maintained medical education thanks to the continuation of medical activities.
Predominance of articles concerning residents
Overall, a majority of the articles reported the impact of the pandemic on residents. This result may reflect the substantial proportion of clinical skills training in residency programmes compared with academic teaching, particularly in technical specialties such as surgery, interventional cardiology or endoscopy. Conversely, thanks to the predominance of theoretical teaching in medical students’ education, continuation of programmes using e-learning, videoconferences, and virtual classes was feasible and efficient.41 46 61 In addition, standardised patients interviews using telehealth format also permitted improvement and assessment of clinical skills among medical students and residents.31 36 46 54 55 57
Predominance of articles concerning academic teaching
This review shows that 88% of the articles reported the use of online courses, suggesting a translation from in-person to virtual classes. Pre-existing courses and educational material probably facilitated the fast implementation of online courses to cope with cancellation of in-person classes due to social distancing. Similarly, free social media and meeting platform software allowed the development of online courses without delay. Conversely, few studies reported the development of new educational tools, such as virtual flipped class room, movies, gaming/quiz competitions….20 33 42 57 Although the pandemic necessitated to find quick and feasible solution for medical education within medical schools and health sciences programmes, this need was mitigated by competing priorities of healthcare delivery as many medical educators are also clinicians. The development of online courses may have been the first step toward a digital learning translation. Time for innovations will come later. Consequently, a systematic review on new educational tools may be performed at the end of the pandemic.
Over-representation of the USA
Among articles included, 61% investigated medical education in the USA. This result is consistent with the overall proportion of medical education articles that originates from North America.70 In addition, medical education departments emerged in the early 60s in the USA, and most medical schools in the USA have a medical education office.71 Conversely, medical education departments first appeared in European medical universities in the early 2000s72 and are not yet widespread in all European universities.
Assessment of the quality of medical education tools during the COVID-19 pandemic
Most included articles were field experience reports. Only 12/60 reviewed articles were observational/interventional studies, the MERSQI could be evaluated in only 4 of them, ranging from 5.5 to 9/18. Furthermore, among these four methodologically acceptable articles three are letters with few details about the evaluation of the educational tool. Moreover, the study population was very small in the survey by Geha et al41 and no detail concerning the evaluation of the educational tool by medical students was given in the survey by Mooney et al.46
This highlights the low level of currently available evidence on the impact of the COVID-19 pandemic on medical education.
Comparison with a systematic review of the COVID-19 medical literature
Liu et al performed a systematic review of all medical literature on COVID-19 published between 1 January and 24 March 2020.73 Some similarities between this review and ours can be found. First the authors highlighted a great number of editorials, commentaries, and opinions in the medical literature, reaching 58% of the articles corresponding to the topic. We found 81% of expert opinions or feedback articles in our review. Second, the lack of methodologically robust studies was also mentioned and was explained by the insufficient time to design such studies. Third, the paucity of technology related articles in the COVID-19 medical literature was emphasised and the same weakness was noticed in our review, with no truly innovative educational tool evaluated in the literature at the time of our review.
This review has several limitations. First, it was carried out too early to include robust study design evaluations of new educational tools. Moreover, specialists highly involved in the clinical care of COVID-19 patients had less time to perform medical education studies or write medical education opinion papers. On the opposite, specialists whose clinical activity was delayed due to the pandemic, such as surgeons, had more time to do so. Second, the new online educational tools include a wide variety of terms that are not necessarily referenced in MeSH and our search may have missed relevant articles. This could also have induced a misclassification of some learning tools. For instance, one article uses ‘forum’ in the title and ‘e-learning programmes’ in the introduction, which correspond to websites with cases, lectures, written material and interactive master classes.21
Our review shows that the disruption of medical education highly impacts the well-being and training of medical students and residents. Implementation of online courses using meeting platforms is a quick and efficient solution to maintain a link between the university and its students. Furthermore, some studies suggested that thanks to anonymous course format, a subgroup of students were more likely to ask questions during remote conference.60 In addition, online courses are suitable for innovative pedagogical solutions such as serious games or reverse pedagogy classes.20 33 42 57 Surgery residents should be particularly supported using simulation to maintain technical training.
Lastly, the COVID-19 pandemic gives the opportunity to all pedagogical chairs to test innovative solutions using all available media. In order to recommend good practices with remote medical education, a rigorous methodological evaluation based on MERSQI criteria is of utmost importance. Future studies should pay attention to strong experimental designs (such as randomisation, control group…) to assess relevant outcomes (with objective measurements, response rate reporting, results beyond descriptive analysis and results on patient/healthcare outcome).8
To conclude, this systematic review has demonstrated that resident’s medical education was highly impacted by the COVID-19 pandemic particularly in surgical specialties. Online courses were the most frequently attempted solution to cope with social distancing constraints although they are not very efficient for the improvement of clinical skills. Medical students’ opinion on pedagogical tools was mostly positive.
In this restricted review including 53 studies, online courses were the most frequently used pedagogical tool.
Virtual reality and simulation tools were used significantly more frequently in surgical specialties compared with medical specialties.
Only three studies reported the assessment of the quality of the pedagogical tools by medical students, using Medical Education Research Study Quality Instrument score and suggested low-quality studies.
Current research questions
The long-term impact on students’ final choice of specialty and career needs to be evaluated.
The delay in skill acquisition will have to be quantified and should be compared between specialties to assess the variability of the impact of the pandemic on medical education.
The limited number of evaluated studies and the low quality of these studies indicate that this restricted review needs to be repeated to include a larger number of more robust studies.
What is already known on the subject?
The COVID-19 outbreak has dramatically impacted medical education, both bedside and academic teaching.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
FC and MB are joint first authors.
FC and MB contributed equally.
OS and AL contributed equally.
Contributors FC, MB, OS and AL designed the study. FC, MB and AL acquired, analysed and interpreted the data. FC, MB and AL wrote the manuscript. OS revised it critically for important intellectual content. FC, MB, OS and AL approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.