Purpose of the study To collect validity evidence for the chest tube insertion (CTI) test mode on the medical simulation application Touch Surgery. This was done by using Messick’s contemporary framework.
Methods Novice, intermediate and experienced participants provided informed consent and demographic information. After familiarisation with the application, they completed the CTI test mode. Validity evidence was collected from four sources: content, response process, relation to other variables and consequences. A post-study questionnaire with 5-point Likert scales assessed the perceived realism, relevance and utility of the assessment. Mean scores of the three groups were compared.
Results A total of 25 novices, 11 intermediates and 19 experienced participants were recruited. Content evidence was collected by an expert in CTI and was based on published literature and guidelines. All users were familiarised with the application, and received standardised instructions throughout the test. Most users rated the simulation as realistic and suitable to assess cognitive skills. Novices received significantly lower (55.9±7.5) test mode scores than intermediates (80.6±4.4) (p<0.001) and experienced participants (82.3±5.3) (p<0.001). There was no significant difference in score between intermediate and experienced participants (p=0.75). Consequences evidence was provided by establishing a pass/fail score of 71% using the contrasting groups method, which resulted in one observed false positive and no false negatives.
Conclusion A robust validity argument was constructed for the CTI test mode, which can be implemented in surgical curricula to assess learners’ cognitive skills prior to hands-on simulation practice.
- medical education & training
- thoracic surgery
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Contributors LDM and JV planned and performed the study and performed statistical analysis. IVH, LD, LK and WW were responsible for direct supervision of the study, with WW being the main guarantor. All authors were responsible for the reporting of the findings, with LDM submitting the manuscript.
Funding LDM is supported and funded by the Special Research Fund (BOF) of Ghent University (BOF.STG.2019.0093). IVH is supported by a Senior Clinical Fellowship (802314N) of the Fund for Scientific Research—Flanders, Belgium.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.