Jump to comment:

• Utility of ML algorithms: Are we predicting the outcome or finding the risk factors.
  Thenmozhi Mani, Marimuthu S and Jeyaseelan L
  Published on: 30 March 2022

• Published on: 30 March 2022

  Utility of ML algorithms: Are we predicting the outcome or finding the risk factors.

  • Thenmozhi Mani, Sr.Demonstrator Christian Medical College and hospital, Vellore, India
  • Other Contributors:
    • Marimuthu S, Associate Research Officer
    • Jeyaseelan L, Professor

  In the diagnosis of ST segment elevation MI, Qin et al (2021) had published an article which wherein the study was meticulously planned, analysed and interpreted in a comprehensive manner (1). The authors have used three ML algorithms to screen variables for prediction and evaluated six algorithms to select the best one that addressed the research question. As stated above the “aim was to use the constructed machine learning (ML) models as auxiliary diagnostic tools to improve the diagnostic accuracy of non-ST-elevation myocardial infarction (NSTEMI)”. Towards the end of a robust analysis, they were able to suggest an ML algorithm XGBoost as the best when compared to other algorithms and Logistic regression model as well.
  In the process of addressing the above aim, they have compiled data retrospectively with a total of 2878 patients from January 2017 to December 2019. Of them 1409 patients were diagnosed with NSTEMI and 1469 patients were diagnosed with unstable angina pectoris. Thus the percentage of NSTEMI in the study was 48.9%. Does it reflect the hospital prevalence of NSTEMI? That is, if 100 patients visit the Cardiology department, what percent of them will have NSTEMI? Supposing that if this is 10%, then would the above identified variables of importance be the same? The probability of a patient having NSTEMI be the same? The ML algorithms are expected to work very well with 50% probability of disease. By not defining the sampling method, the process of ba...

  Show More

  In the diagnosis of ST segment elevation MI, Qin et al (2021) had published an article which wherein the study was meticulously planned, analysed and interpreted in a comprehensive manner (1). The authors have used three ML algorithms to screen variables for prediction and evaluated six algorithms to select the best one that addressed the research question. As stated above the “aim was to use the constructed machine learning (ML) models as auxiliary diagnostic tools to improve the diagnostic accuracy of non-ST-elevation myocardial infarction (NSTEMI)”. Towards the end of a robust analysis, they were able to suggest an ML algorithm XGBoost as the best when compared to other algorithms and Logistic regression model as well.
  In the process of addressing the above aim, they have compiled data retrospectively with a total of 2878 patients from January 2017 to December 2019. Of them 1409 patients were diagnosed with NSTEMI and 1469 patients were diagnosed with unstable angina pectoris. Thus the percentage of NSTEMI in the study was 48.9%. Does it reflect the hospital prevalence of NSTEMI? That is, if 100 patients visit the Cardiology department, what percent of them will have NSTEMI? Supposing that if this is 10%, then would the above identified variables of importance be the same? The probability of a patient having NSTEMI be the same? The ML algorithms are expected to work very well with 50% probability of disease. By not defining the sampling method, the process of balancing 50:50 probability of disease and no disease might provide results that cannot be used in real life situation. As known in clinical epidemiology the positive predictive value of getting disease depends on the prevalence of disease. Therefore, the results have to be used with caution from this paper.

  The variables (features) of importance help us to order the hierarchy of the study variables. However, this information is still not useful to be utilized such as relative risk or odds ratio. As suggested by Vollmer (2020) that clinically relevant research using ML and AI algorithm is often limited by transparency, reproducibility, ethics, and effectiveness. Unfortunately, the transparency is not available in algorithms such as SVM and XGBoost etc (2). While the patients and healthcare professionals require clinical prediction models to make decisions, the ML algorithms are “black box models” and the regression coefficients are not available. Though the ML algorithm provide the probability of getting the outcome of interest using appropriate codes in R, Python software, can we develop a prediction model similar to Glasgow coma score or APACHE II score using ML algorithms? As the ML algorithms are not easily computable like logistic regression etc., to predict the probability, we wonder the utility of using the variables of importance only. Do we need a model to assess the probability of getting outcome of interest besides the variables of importance with weights or just the variables of importance alone? How much trade off we would like to do in choosing between ML algorithms versus Logistic or Cox proportional hazard regression models?

  Reference:
  1. Qin L, Qi Q, Aikeliyaer A, Hou WQ, Zuo CX, Ma X. Machine learning algorithm can provide assistance for the diagnosis of non-ST-segment elevation myocardial infarction. Postgraduate Medical Journal 2022 Feb 16 Available from: https://pmj.bmj.com/content/early/2022/02/15/postgradmedj-2021-141329

  2.Vollmer S, Mateen BA, Bohner G, Király FJ, Ghani R, Jonsson P, et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ. 2020 Mar 20;368:l6927.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top