Table 3

Study outcome measures and main findings

Lead author,
year,
Institution,
country
Outcome assessmentMain summarised findingsKirkpatrick criteria measuredKirkpatrick level
Back,26
2019,
Bundeswehr Hospital, Germany
Precourse and postcourse examination and questionnaire assessing disaster medicine knowledge and interest in disaster medicine preparedness assessed by questionnaire.This disaster course significant increased knowledge of disaster medicine after course from 56% to 72% (p<0.001). Students strongly interested in disaster medicine increased after the course, from 63% to 80%. More than 80% agreed or strongly agreed they were satisfied with the course, and their increase in knowledge.Knowledge, attitude1, 2A, 2B
Bajow,13
2016,
King Fahd Security College, Saudi Arabia
Precourse and postcourse examination assessing disaster medicine knowledge. Questionnaire sent 1.5 years after course to assess perceived effect of course.There was a significant increase in knowledge of disaster medicine after course from 41% to 68% (p<0.0001), independent of medical student year. Medical student year. Medical students once graduated felt less stressed and confident when handling emergencies 1.5 years after the course. More than 70% found the course interesting.Knowledge, attitude1, 2B, 3
Carney,27
2011,
Case Western Reserve, Harvard, Colorado and Vermont Colleges of Medicine, USA
Postcourse questionnaires assessing perceived understanding of public health system and response to public health threats, perceived understanding of successful pandemic/emergency preparedness response. Examination assessing knowledge of pandemic/emergency preparedness.Overall, 69% agreed or strongly agreed they had a better understanding of the public health, 82% agreed or strongly agreed they had a better understanding of successful preparedness for a pandemic or emergency, and 70% correctly answered questions on pandemic/emergency preparedness after course. 100% agreed or strongly agreed that physicians are best prepared by having written plans and undertaking preparedness exercises after the course.Knowledge, attitude2A, 2B
Chernock,28
2019,
Rutgers New Jersey Medical School, USA
Postcourse questionnaire assessing perceived confidence using skills applying tourniquet.92% agreed or strongly agreed they were confident using the tourniquet if required after course. 98% agreed or strongly agreed they had learnt the basics of bleeding control. 89% would recommend the course.Attitude, skill1 to 2B
Goolsby,15
2014,
University of the Health Sciences, USA
Postcourse questionnaire assessing perceived confidence at assessment and procedures in a combat casualty situation and perceived preparedness at managing combat casualties.The majority of students feel more confident and better prepared to assess and perform procedures in a combat casualty situation after course. The majority of students preferred the high-fidelity simulation to their normal learning environment.Attitude, skill1 to 2A
Ingrassia,29
2014
CRIMEDIM, Italy
Precourse and postcourse examination assessing disaster medicine knowledge. Triage accuracy was measured in a disaster simulation assessed by an examiner.There was a significant improvement in knowledge of disaster medicine after course from 40% to 83% (p<0.01), and a significant improvement in triage accuracy in the disaster medicine simulation after course from 45% to 78% (p<0.01). The majority of students felt that disaster medicine should be part of their curriculum and evaluated the course highly.Knowledge, skill1 to 2B
Kaji,30
2010,
University of California, USA
Postcourse oral examination of disaster medicine knowledge.All participants obtained a scores of >90% obtained on examination after course. All students rated the course 100% on a five-point Likert type scale.Knowledge1 to 2B
Lei,31
2019,
UTHealth McGovern Medical School, USA
Precourse and postcourse examination assessing disaster medicine knowledge and questionnaire assessing perceived willingness and preparedness to help bleeding individual.All medical students passed the test on haemorrhage control after the intervention, compared with 73% prior to the course. Willingness of medical students to help a bleeding volunteer increased from 93% to 99% and preparedness increased from 19% to 98%.Knowledge, attitude, skill2A, 2B
Lin,32
2009,
University of Illinois, USA
Postcourse assessment of ability to perform bag valve mask and examination assessing knowledge of methods for ensuring adequate ventilation.All students were able to satisfactorily perform bag-valve mask technique after the course. The majority of students knew proper positioning in non-trauma (93%) and trauma cases (72%), and ventilation rates (86%), and technique to ensure adequate seal (63%). However, only 29% knew how to assess adequate ventilation.Knowledge, skill2B
Marcus,33
2019,
University of Toledo, USA
Precourse and postcourse examination assessing tourniquet application knowledge, and questionnaire on perceived competency in applying tourniquet.There was a significant improvement in knowledge of tourniquet application on test. Likert score increased from 2.3 to 4.4 after course (p<0.001). There was also a significant improvement in confidence when applying tourniquet (p<0.001).Knowledge, skill2A, 2B
Marshall,34
2008,
University of Hawaii School of Medicine, USA
An educationalist observed problem-based learning sessions and administered precourse and postcourse questionnaires on subjective knowledge in addition to online surveys on postcourse bioterrorism knowledgeProblem-based learning is effective in educating both medical students and community-based health professionals from different disciplines about issues related to pandemic preparedness in addition to allowing multidisciplinary communication and collaboration.Knowledge, attitude2B
Myong,35
2016,
The Catholic University of Korea, Republic of Korea
A qualitative fit testing tool (3M-FT10 kit) was used to assess for a pass fail of adequate respiratory protection fitting precourse and postcourse.This course demonstrated effective teaching of respiratory protection fitting in medical students, with the proportion of individuals passing the test being 30% (n=15) before the programme and 74% (n=37) after. This may reduce the risk of infection in medical students working in the hospital with at risk patients and may allow medical students to teach this skill.Skill2B
Padaki,36
2018,
Department of Emergency Medicine, Christiana Care Health System, USA
Presimulation and postsimulation training questionnaires were administered based on knowledge an anonymous participant feedback was solicited for purposes of course improvement.This course demonstrated a practical, low-fidelity simulation-based curriculum for education on in-flight medical emergencies. Simulation training significantly increased student performance, from a mean pretest score of 75.6% to a mean post-test score of 87.0%.Knowledge2B
Parrish,37
2005,
The Texas A&M College of Medicine, USA
Participants were asked to undertake precourse and postcourse surveys assessing their knowledge of bioterrorism and a post-test survey assessing their attitudes towards preparedness for bioterrorism.After this course, students were more favourable in their attitudes towards their professional preparedness and the local/state government preparedness for a bioterrorist event which were rated with a mean Likert score of 4.4. There was a statistically significant improvement (8.6 to 10.5) in students’ knowledge of disaster preparednessKnowledge, attitude2A, 2B
Patel,38
2016,
Case Western Reserve University School of Medicine, USA
Participants undertook precourse and postcourse surveys assessing knowledge of acronyms Participants undertook precourse and postcourse surveys grading self-assessed preparedness for medical disasters.An online elective in disaster preparedness resulted in students becoming more familiar with acronyms such as EMA (8% pre to 90% post), gaining a better understanding of organisations such as the American Red Cross (36% pre; 73% post), gaining triage knowledge (START triage 15% pre; 71% post). Similar proportions of students would volunteer in disaster scenarios precourse and postcourse (94% pre; 93% post).Knowledge, attitude2A, 2B
Rivkind,39
2015,
Hadassah-Hebrew University Medical Center, Israel
Students were observed thorough multidisciplinary debriefing based on video footage and action photographs. Students received feedback on communication and interaction under stress, triage decisions and clinical management.Over a 3-year assessment period with data on 309 participants out of 490 in total, the mean knowledge pretest was 54% (12.7%) compared with 68% (10.2%). On an assessment scale of 1 to 20, students in the 2012 cohort scored the course highly for its general assessment, with high results in trauma knowledge gained mean 18.4 (1.2), assessment of self-preparedness 15.9 (3.1) and technical skills acquired mean 17.0 (2.4).Knowledge, attitude, skill1, 2A, 2B
Scott40
2010,
Medical University of South Carolina, USA
Students in the 2008 class took the pretest to survey basic knowledge and assess learning of the didactic material immediately before the 90 min case-based lecture, and the post-test immediately after the lectures. The 2009 class could take their pretest via an e-learning tool up to several days before the class, and the post-test was available online for 3 weeks after completion of the course.Over a 2-year assessment period the first year cohort's post-test knowledge scored improved from 3.8/10 (below average to average) compared with 7.6/10 (average to above average) and the second year's post-test scores improved from 2.5/5 (average) before and 3.8/5 (above average). In the first year cohort he average overall rating for the experience was 4.9/5, and 100% of the respondents recommended the class for next year’s students allowing it to continue.Knowledge, attitude1 to 2B
Scott,41
2012,
Medical University of South Carolina, USA
Participants undertook precourse and postcourse assessment developed to meet learning objectives of the course. Self-assessment of personal capability and comfort to handle a disaster and multiple choice questions of knowledge and subjective skill were undertaken.Most (70%) of the trainees considered their emergency preparedness knowledge and skill as average or below average before the training experience. After the curriculum, 100% of trainees considered their emergency preparedness knowledge and skill above average, and 90% would recommend the course to other healthcare workers.Knowledge, skill1 to 2B
Scott,14
2013,
Medical University of South Carolina, USA
Participants undertook an online precourse and postcourse assessment developed to meet the learning objectives and competencies of the course in addition to giving post-test feedback on the implementation of the course.In discrete knowledge, subjective knowledge and skills all participants demonstrated significant improvements in their postcourse test results when compared with pre-test. Course evaluation was performed, and it was found that students would recommend this course (median 92.5%), whether the course was feasible (median 82.5%) and overall evaluation (94.5%).Knowledge, skill1 to 2B
Silenas,16
2008,
The Texas A&M College of Medicine, USA
Students answered Likert type scales to assess the extent to which the objectives and understanding of key concepts had been accomplished. Written and verbal comments from the students and facilitators about their experience were gathered.Sixty-six medical students completed the knowledge test before and again 4 days after the Avian Influenza exercise. The lowest scores for knowledge were best and all tested knowledge areas except one (endemic influenza as a public health issue) decreased postcourse. The course received mixed ratings which overall were positive (33%), undecided (13%) and negative (54%).Knowledge.1 to 2B
Vincent,42,
2008,
Telehealth Research Institute, University of Hawaii, USA
Students answered precourse and postcourse self-confidence questions on a five-point Likert scale with points labelled 'never' to 'always’ in addition to giving evaluation on the course implementation and use of virtual reality and between exercise virtual reality scores.Students became more confident that their patients would consider them effective first responders (p=0.006), more confident in prioritising treatment (p=0.001), more confident in prioritising resources (p=0.001) and more confident in identifying high-risk patients (p=0.008). Students rated the simulation phase of the course highly and favourably rated on a Likert seven-point scale: pace (4.2±0.39 too slow/too fast), level of difficulty (4.5±0.83 too easy/too hard) and relevance (6.5±0.61 agree/disagree).Attitude1 to 2A
Vincent,43
2009,
Telehealth Research Institute, University of Hawaii, USA
Students answered precourse and postcourse self-confidence questions on a five-point Likert scale with points labelled 'never' to 'always’ in addition to giving evaluation on the course implementation and use of simulation and between exercise simulation scores.Following this course students became more confident that their patients would consider them effective first responders (p<0.001), more confident in prioritising treatment (p<0.001), more confident in prioritising resources (p<0.01) and more confident in identifying high-risk patients (p<0.01). The students rated the simulation phase of the course highly and favourably rated on a Likert seven-point scale pace (4.2 too slow/too fast), level of difficulty (4.0 too easy/too hard) and relevance (6.8 agree/disagree).Attitude1 to 2A
Wiesner,44
2018,
Georgetown University School of Medicine, USA
Students answered precourse and postcourse knowledge-based tests with a maximum score achievable being 10. No outcomes were gathered on the skills based workshops teaching skills such as suturing and decontamination.Students displayed significant improvement in their disaster medicine knowledge through completion of the course, with an improvement being demonstrated between mean precourse test score 5.3 (1.1) and mean postcourse test score 8.0 (1.0). The mean improvement in scores for all students on this course was 2.7 (p<0.0001, 95% CI 2.3 to 3.1).Knowledge, skill2B