Introduction

Perioperative morbidity and mortality is a significant public health issue, due to its impact on patients' short and long term survival, and also resource utilisation within the health service. Surgical complications occur in 3–17% of patients1 2; in the general population, surgical mortality is approximately 0.5%, but in elderly patients undergoing emergency surgery the UK mortality rate is >12%.3

As clinicians we strive to find accurate methods to stratify patient risk from undergoing surgery. Clinical risk scores tend to focus on patient history, examination, preoperative investigations, surgical severity, and intraoperative events. Surgical risk may also be assessed using measures of functional capacity, such as the Duke Activity Status Index, and more objectively, using cardiopulmonary exercise testing (CPET).4 5 There is also increasing interest in biological markers of cardiorespiratory health or inflammation which could be used to predict perioperative outcome such as N-terminal brain natriuretic peptide (NT-BNP)6 7 and high sensitivity C reactive protein (hsCRP).8 9 However, while both CPET and serum biomarkers show promise as risk prediction tools, neither are widely available, and both require further validation in large pragmatic multicentre studies.

Scoring systems have the advantage that they are usually cost neutral, and can be performed at any time of the day or night. Patients presenting for emergency surgery are at high perioperative risk,3 and it may be possible to predict the outcomes of such patients using these openly available, relatively quick scoring systems which are not subject to complex interpretation.

The ideal risk prediction model would be one that is simple, reproducible, accurate, objective, and available to all patients. Many hospitals lack the resources to run expensive tests, so ideally it should be cheap, and possible to perform at the bedside. As the purpose of the scoring system is to define an individual patient's risk before he or she undergoes surgery, a model that is based entirely on preoperative risk factors would be more beneficial than those which include intraoperative and postoperative variables.

In the course of this article we shall describe the different scoring systems that have been developed and are used to predict risk for non-cardiac surgery, highlighting their advantages and disadvantages, and how their accuracy has be shown to vary in a number of different studies.

Risk stratification scoring systems

Risk stratification scoring systems can be classified into those estimating population risk, such as the American Society of Anaesthesiologists Physical Status score (ASA-PS), and those estimating individual risk. Risk scores developed to calculate individual risk can be subdivided into those which are designed to predict cardiac morbidity and mortality such as the Lee Revised Cardiac Risk Index (RCRI), and those which predict generic morbidity and mortality. These include scores which use solely preoperative risk factors—for example, Charlson Index—and those looking at a combination of preoperative, intraoperative, and postoperative factors—for example, Physiological and Operative Severity Score for the enUmeration of Morbidity and Mortality (POSSUM). We shall now consider each of these in turn.

Population risk: American Society of Anaesthesiologists' physical status score

The ASA-PS uses preoperative physical fitness to categorise patients subjectively into five subgroups. The score was devised in 1941 originally as a statistical tool for retrospective analysis of hospital records.10 It has been revised on several occasions and in 1963 the five point scale most commonly known was first reported.11 There is now an additional category for patients whose organs are being removed for donor purposes (table 1).

Underlying fitness is an important predictor of survival from surgery, and the ASA-PS score has been shown to correlate with outcome in a number of different settings.12–23 It is simple, easy to understand, and commonly used as part of the preoperative assessment. It is also an easy and useful tool to assist descriptions of workload and ‘anaesthetic risk’ for audit and research purposes.

However, the ASA-PS system has a number of limitations. It does not consider the preoperative optimisation of the patient, the surgery planned, or the proposed postoperative care—that is, critical care unit admission—and it makes no adjustments for age, sex, weight, or pregnancy. In this sense it does not give a prediction of risk for an individual patient or operation. Wolters' study found a population mortality of 0.1% for ASA I patients, and an 18.3% mortality for patients graded ASA IV.24 However, more recent work demonstrated that the ASA-PS was poorly predictive of individual patient outcome: while it correctly predicted an uncomplicated course in 96% of patients, complications were correctly predicted in only 16% of patients in whom they occurred (positive predictive value 57%, negative predictive value 80%).25 Similar results were found in a more recent study which looked at infective complications and comorbidity in patients undergoing total knee replacement.26

Individual risk: cardiovascular morbidity and mortality

In 1977 Goldman et al developed a cardiac risk index using nine different preoperative variables.27 Since then, multiple cardiac risk indices have been developed, and there have been a number of guidelines which recommend their use for preoperative cardiac evaluation.28 29

Probably the most widely used cardiac risk index currently was developed by Lee et al in 1999, by revising Goldman's criteria for cardiac risk and using six independent variables.30 A patient is considered to be high risk when they have more than two risk factors. A recent systematic review evaluated the ability of the revised cardiac risk index (RCRI) to predict cardiac complications and mortality after major non-cardiac surgery across different populations and settings.31 Their conclusion was the RCRI was moderately good at discriminating patients who developed complications from those whose clinical course was uneventful after mixed (non-vascular and vascular) non-cardiac surgery. It was less accurate at discriminating cardiac events after vascular non-cardiac surgery and less accurate at predicting mortality. Acknowledged limitations of the review included the statistical and clinical heterogeneity and generally low methodological quality of the included studies, and their varied definitions of ‘cardiac events’31 (box 1).

There is ongoing work on adapting and improving the predictive accuracy of the RCRI. A recent prospective, single centre study assessed the predictive power of NT-proBNP, CRP, and RCRI in combination for the risk of a perioperative major cardiovascular event, and found that the predictive power of the RCRI could be improved significantly by the addition of CRP and NT-proBNP to RCRI (adjusted relative risk (RR) 4.6) (p<0.001).32

While cardiac morbidity is important, it is relatively uncommon compared with other types of perioperative complication, which may have significant implications for patients both in the short and long term.33 34 There is therefore a requirement for a clinical risk score which may predict generic morbidity and mortality with accuracy. Existing systems which are used for this purpose are now discussed.

Generic morbidity and mortality: preoperative risk factors alone

The Charlson Index was developed in 1987 and was based on the 1 year mortality data from 604 medical patients admitted to a New York hospital. The index was then validated for its ability to predict death in a cohort of 685 breast cancer patients.35 The authors went on to validate an age–comorbidity index (Charlson Age Comorbidity Index, CACI) in a non-cardiac cohort of patients to predict long term outcome. Two hundred and twenty-six patients undergoing elective surgery were enrolled in the study, all of whom had hypertension or diabetes. These patients were followed for at least 5 years postoperatively. The estimated RR of death for each comorbidity rank was 1.4 and for each decade of age was 1.4. When age and comorbidity were represented as a combined score the estimated RR for each combined unit was 1.45. The estimated RR of death from an increase of one in the comorbidity score was approximately equal to that from an additional decade of age36 (table 2).

It has since been validated for predicting inpatient morbidity and mortality in a number of different surgical cohorts. A study of lung cancer patients37 identified gender, CACI score 3–4, chronic obstructive pulmonary disease (COPD), and prior tumour within the last 5 years as predictors for major complications. Charlson scores of 3–4 maintained statistical significance in multivariate regression analyses (OR 9.8, 95% CI 2.1 to 45.9). Ouellette et al also found the CACI to be a predictor of in-hospital morbidity, duration of hospital stay and mortality following colorectal surgery,38 and it has helped to calculate mortality following cardiac surgery.39 The CACI has also, in certain patient groups, been found to predict ‘long term’ mortality.40 41

Following head and neck surgery,21 and radical prostatectomy,42 the Charlson Index has a predictive ability comparable to that of the ASA-PS when ascertaining morbidity and mortality. However, the ASA-PS was more accurate in its calculation than the Charlson Index in a cohort of patients undergoing liver resection.43

This series of papers represents a relatively large and diverse experience with the Charlson Index, mainly confirming its predictive validity. However, disadvantages of this scoring system are the lack of information regarding the surgical procedure, and the subjectivity of the patients' comorbidity assessment, which may potentially lead to error.

Generic morbidity and mortality: perioperative risk factors

The Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity (POSSUM) was developed by Copeland et al in 1991 as a scoring system for surgical audit,44 where they used multivariate logistic regression to identify the most significant of 48 physiological variables and 12 operative and postoperative variables for the prediction of the 30 day morbidity and mortality rates. The final POSSUM system incorporates 12 variables to assess physiological status and six ‘surgical’ variables, resulting in an 18 component score. In order for perioperative risk to be calculated, the sum of the physiological and surgical variables are entered into two mathematical equations which are used to calculate the risk of morbidity and mortality (table 3).

A potential drawback in using this scoring system to assess risk is the time at which the operative variables are obtained. The physiological variables are documented before the commencement of surgery, and include the patient's clinical symptoms, signs elicited, haematology and biochemistry results, and an assessment of the electrocardiogram. When it comes to collecting the surgical data, this can be delayed for a prolonged period of time, as included in the score are the number of subsequent operations within 30 days, and presence of malignancy. This causes obvious difficulties in using this as the sole tool to make decisions on the appropriateness of surgery, as a number of variables will not be available until the surgery is complete.

POSSUM was originally developed and validated in a single UK district general hospital, in a study which included emergency and elective procedures in patients undergoing urological, gastrointestinal, vascular, and hepatobiliary surgery.44 Subsequently, this system has also been used in comparisons of surgeons,45 resource utilisation,46 and to compare surgical outcomes in different countries.47 While some studies have found confirmatory evidence that POSSUM predicts individual patient risk of morbidity and mortality,48–50 others have found a significant overestimation of mortality using this score.51–57 This is the result of using logistic regression to predict risk, as the lowest possible mortality risk is 1.08%.58

Using alternative risk equations, but the same physiological and surgical variables, a new risk model was developed in Portsmouth (P-POSSUM), and validated in a large single centre cohort,59 and since then has been shown to predict in-hospital mortality more accurately than POSSUM.48 56 57 60–64 However, P-POSSUM has no morbidity prediction equation, as a result of the original authors' lack of confidence in the reporting of perioperative complications.59 Subsequent studies have shown P-POSSUM to both over-predict54 64 and under-predict mortality48 61 65 in different settings.

Large cohorts of patients have been used to validate variations of POSSUM for specific surgical groups, including Cr-POSSUM (colorectal surgery)54 63 65 66 and V-POSSUM (vascular surgery),50 which are more sensitive and specific for predicting patient outcome following these operations. However, a recent systematic review of the use of different POSSUM models in oesophageal–gastric surgery found that the P-POSSUM was superior to both the original POSSUM and the surgery specific O-POSSUM model for the prediction of postoperative mortality.67 These data highlight the considerable variation in the predictive precision of these models in different surgical cohorts. One of the reasons for this variation may be the inclusion of a number of subjective variables such as jugular venous pressure measurement and chest radiograph interpretation, which may introduce error in risk score calculation. Nevertheless, POSSUM and its subsequent variations remain the most internationally validated risk scoring system for predicting individual patient risk.

Biochemical and haematological outcome models

The Biochemistry and Haematology Outcome Model (BHOM) was developed by Prytherch et al68 in 2003, and data collected includes age, sex, mode of admission (eg, emergency or elective), physiological parameters (haemoglobin, white cell count, sodium, potassium, urea), BUPA operative severity score, and 30 day mortality. The BUPA operative severity score is used in the private sector (UK) as an indicator of surgical complexity and workload. The BHOM uses information which is easily obtainable from most hospital information technology systems to calculate predictive mortalities.69 70 As previously stated, a risk assessment scoring system should ideally have few variables, which are simple to collate and therefore make the score easy to implement in daily practice. It should be available for every patient and have limited potential inter-observer variability. BHOM uses objective data, which are easily available from a single blood sample. It is therefore feasible to collect this for all patients as part of routine care. However, BHOM is one of the less widespread outcome models used presently, and is yet to be validated in large multicentre cohorts including patients undergoing different types of surgery.

The surgical Apgar score

The Apgar score was originally developed as a powerful tool in obstetrics to rapidly assess and gain feedback on a newborn's condition.71 The surgical Apgar score was piloted using general and vascular patients where the score was significantly associated with the occurrence of major complications or death within 30 days of surgery (p<0.001).72 The 10 point score is calculated from the estimated blood loss, lowest heart rate, and lowest mean arterial pressure during an operation. It uses routinely available data, and has been to shown to identify immediately and effectively those patients at higher and lower risk of complications and death postoperatively.73 Its limitations include its lack of validity in other cohorts of patients, lack of comparison between different institutions, and the potential for imprecision resulting from ‘estimating’ blood loss. As a consequence, it may be considered one of the less applicable scoring systems in current clinical practice.

An American approach: the National Surgical Quality Improvement Program

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) is the first nationally validated, risk adjusted, outcomes based programme used to measure and improve the quality of surgical care in the USA.74 A prospective, validated database is used to quantify 30 day risk adjusted surgical outcomes. A comparison of outcomes can then be made for all the hospitals within the programme. ACS NSQIP collects data on 136 variables for patients undergoing major surgical procedures, including preoperative risk factors, intraoperative variables, and 30 day postoperative morbidity and mortality. The data are collected, validated, and submitted by a trained surgical clinical reviewer at each site. Using logistic regression, risk adjusted 30 day morbidity and mortality outcomes are made for each participating hospital on an annual basis. Both generic and surgery specific models are generated, and outcomes are reported as observed versus expected (O/E) ratios. The ACS NSQIP database therefore allows hospital specific adjustments to surgical risk to be incorporated, by allowing the categorisation of hospitals performing better or worse than expected.

The unique aspect of this initiative is the generation of bespoke models which are responsive to changes in population characteristics over time, and therefore are more precise. This was demonstrated in a recent study which compared an ACS-NSQIP model for colorectal surgery with existing validated models including the CR-POSSUM and the Association Francaise de Chirugie colorectal model. The ACS NSQIP colorectal risk calculator demonstrated far higher predictive precision than existing models in both development and validation cohorts.75 Despite the advantages of such initiatives, their application internationally is limited by the resources required to collect and record multiple variables prospectively and accurately.

Conclusions and future work

Clinicians within the perioperative team utilise risk prediction scores to help make important, informed decisions to optimise an individual's perioperative management. Accurate preoperative determination of risk is also important to enable truly informed consent for patients who may be at high risk, particularly for operations where non-surgical interventions may be an alternative option. Looking at the available research it is clear that there are limitations to many of the risk stratification scoring systems that are currently available.

With clinical risk scores using subjective variables, the method of data collection, and therefore the accuracy of the data collected, may influence the predictive accuracy of a particular risk model. Even relatively simple systems such as the ASA-PS have been demonstrated to be subject to inter-observer variation in their assessment,76 and there is a risk of diagnostic inaccuracy with systems which use administrative databases as a source of information, or untrained medical staff as data collectors as opposed to prospective data collection by trained staff.77 78 The use of objective variables can improve reliability and precision of risk scoring systems. The technology already available in hospitals may aid future collection of routine variables which can be incorporated into an objective dataset to help predict risk. If this is then combined with biomarker assays and assessment of functional capacity, improved performance may be possible.

Further work is required to improve the identification of patients at high risk of perioperative morbidity. Such work should include both research required to identify objective risk factors which are significant on multivariate analysis and may be combined in a scoring system with high predictive precision, and also investment in resources enabling both objective and subjective variables to be recorded and collected accurately. As medical and surgical practice changes with time, it is likely that the predictive precision of risk scoring systems may also change: the implications of comorbid illness may not be the same now as during the 1990s when systems such as the RCRI and POSSUM score were first validated. Furthermore, standards of healthcare and perioperative practice will differ between institutions and countries, with the consequence that a ‘one size fits all’ solution may not be possible. Therefore, the ideal risk prediction system, in addition to consisting of predominantly preoperative variables, should also be responsive to changes in clinical management over time. If subjective variables are used, then these should be assessed by staff trained in such assessment. The ACS-NSQIP initiative approaches these ideals; however, for the rest of the world, while existing systems such as the RCRI and POSSUM scores are helpful, further work aimed at evaluating and refining them in different populations and healthcare systems is required.

Multiple choice questions (true (T)/false (F); answers after the references)