Statistics from Altmetric.com
Medical knowledge is an essential unit of the spectrum of entrustable professional activity for physicians, and knowledge assessment for the certification of medical licencing exists in many countries. To be registered as physicians, Japanese medical students must succeed in the National Medical Practitioners Qualifying Examination (NMPQE), which consists of multiple-choice questions that evaluate competence in basic clinical knowledge needed during the mandatory 2-year residency training. Such training comprises super rotations of internal medicine, emergency medicine, surgery, obstetrics and gynaecology, paediatrics and psychiatry. It is important that qualifying examinations in medicine be validated based on experience acquired in actual clinical practice; for example, the content of the maintenance of certification examination administered by the American Board of Internal Medicine is based on conditions encountered in clinical practice.1 However, the validity of the Japanese NMPQE has not yet been evaluated; thus, to specifically identify conditions not covered by the NMPQE, we compared the contents of the NMPQE for the fiscal year 2016 with clinical conditions frequently encountered by physicians during junior residency, that is, clinical training in postgraduate years 1 and 2.
We collected data on conditions encountered by doctors during junior residency in the Urasoe General Hospital and other affiliated hospitals. Thirteen resident physicians had passed the NMPQE for the fiscal year 2016 and had completed the mandatory 2-year junior residency training programme at the Urasoe General Hospital. We identified disease conditions encountered by these residents based on discharge summaries submitted by them to the electronic health records of the training hospitals.
Morphological analysis was used to identify diseases or conditions from discharge summaries and compare them with question items, choices and answers in the NMPQE. In morphological analysis, a sentence is reduced into morphemes, and their grammatical attributes (eg, parts of speech and conjugated forms) are determined. Disease conditions not mentioned in the NMPQE but encountered at least 10 times in the discharge summaries were listed; conditions encountered in outpatient settings were excluded by two general internal medicine specialists. A representative illustration of morphological analysis is provided in figure 1.
We identified 8016 disease conditions listed in discharge summaries, while only 5572 disease conditions were included in the NMPQE. Table 1 lists the disease conditions (n=20) that were not mentioned in the NMPQE but were listed at least 10 times in the discharge summaries.
Our results suggest that the NMPQE did not cover several disease conditions commonly encountered by junior residents, although the fundamental aim of the junior residency training programme is acquiring basic clinical competency on common diseases, integrating clinical knowledge and skills, and applying these to patients as needed. For instance, hip fracture did not appear in the NMPQE; nevertheless, it is a common condition among elderly Japanese patients that must be comanaged by hospitalists or internists with orthopedists to ensure the quality and safety of treatment. Moreover, the NMPQE did not cover important differential diagnoses of certain diseases; for example, necrotising fasciitis (NF) was included in the NMPQE (data not shown), whereas cellulitis, which should be part of the differential diagnosis for NF, was not mentioned in the examination. These results suggest that the NMPQE did not provide due importance to common differential diagnoses, which may lead to discrepancies between knowledge required for graduating medical school and common conditions encountered by junior residents during postgraduate training.
With progress in the internationalisation of healthcare workers, including physicians, the quality of practice and medical education for medical professionals need to be warranted.2 The World Federation for Medical Education (WFME) has proposed a framework for a global standard of quality improvement in medical education, and the importance of clinical knowledge is mentioned as an important component.3 In the UK, the General Medical Council plans to implement the UK Medical Licensing Assessment, a large-scale licencing examination for medical graduates. This initiative aims to collect data about the deviation of quality of performance related to differences in the unique curricula of medical colleges.4 Although the causal evidence should be identified, a higher score on a large-scale licencing examination and better clinical performance may be correlated.5 Thus, the validity of a licencing examination can be crucial for assessing clinical performance. Clinical knowledge needs to be validated in this context, and our methodology for assuring content validity may be useful for assessing consistency between clinical practice and the educational content that is covered in a licencing examination. Further, the WFME stresses the importance of local needs and the context of each nation when implementing the framework.3 Assessing content validity in each nation may be warranted, and theoretically, morphological analysis in various languages may be useful. This can be implemented for countries that have their own national medical licencing examination.
The present study is limited by selection bias as the residents belonged to a single hospital programme. Further, we analysed the NMPQE of only a single year.
In conclusion, the medical licencing examination in Japan appears to not include several disease conditions commonly encountered by junior residents during postgraduate training. Concordance between the NMPQE and conditions encountered during junior residency should be enhanced to comprehensively cover common diseases encountered during junior residency training.
The authors thank Enago (www.enago.jp) for English language review.
Contributors TS, KK and YN designed this study in whole and drafted the manuscript. TS and YN contributed to data collection. SK contributed to statistical analyses in this study. All authors contributed to provide advice on the interpretation of the results. YT revised the manuscript critically for important intellectual content and approved finally the manuscript submitted. All authors read and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.