Developing and validating subjective and objective risk-assessment measures for predicting mortality after major surgery: An international prospective cohort study
Autoři:
Danny J. N. Wong aff001; Steve Harris aff003; Arun Sahni aff001; James R. Bedford aff001; Laura Cortes aff002; Richard Shawyer aff004; Andrew M. Wilson aff005; Helen A. Lindsay aff005; Doug Campbell aff005; Scott Popham aff006; Lisa M. Barneto aff007; Paul S. Myles aff008; ; S. Ramani Moonesinghe aff001
Působiště autorů:
UCL/UCLH Surgical Outcomes Research Centre, Centre for Perioperative Medicine, Department for Targeted Intervention, Division of Surgery and Interventional Science, University College London, London, United Kingdom
aff001; Health Services Research Centre, National Institute of Academic Anaesthesia, Royal College of Anaesthetists, London, United Kingdom
aff002; Bloomsbury Institute of Intensive Care Medicine, University College London, London, United Kingdom
aff003; Lay representative, United Kingdom
aff004; Auckland City Hospital, Auckland District Health Board, Auckland, New Zealand
aff005; Gold Coast University Hospital, Southport, Queensland, Australia
aff006; Wellington Regional Hospital, Capital & Coast District Health Board, Wellington, New Zealand
aff007; Department of Anaesthesiology and Perioperative Medicine, The Alfred Hospital, Melbourne, Victoria, Australia
aff008
Vyšlo v časopise:
Developing and validating subjective and objective risk-assessment measures for predicting mortality after major surgery: An international prospective cohort study. PLoS Med 17(10): e32767. doi:10.1371/journal.pmed.1003253
Kategorie:
Research Article
doi:
https://doi.org/10.1371/journal.pmed.1003253
Souhrn
Background
Preoperative risk prediction is important for guiding clinical decision-making and resource allocation. Clinicians frequently rely solely on their own clinical judgement for risk prediction rather than objective measures. We aimed to compare the accuracy of freely available objective surgical risk tools with subjective clinical assessment in predicting 30-day mortality.
Methods and findings
We conducted a prospective observational study in 274 hospitals in the United Kingdom (UK), Australia, and New Zealand. For 1 week in 2017, prospective risk, surgical, and outcome data were collected on all adults aged 18 years and over undergoing surgery requiring at least a 1-night stay in hospital. Recruitment bias was avoided through an ethical waiver to patient consent; a mixture of rural, urban, district, and university hospitals participated. We compared subjective assessment with 3 previously published, open-access objective risk tools for predicting 30-day mortality: the Portsmouth-Physiology and Operative Severity Score for the enUmeration of Mortality (P-POSSUM), Surgical Risk Scale (SRS), and Surgical Outcome Risk Tool (SORT). We then developed a logistic regression model combining subjective assessment and the best objective tool and compared its performance to each constituent method alone. We included 22,631 patients in the study: 52.8% were female, median age was 62 years (interquartile range [IQR] 46 to 73 years), median postoperative length of stay was 3 days (IQR 1 to 6), and inpatient 30-day mortality was 1.4%. Clinicians used subjective assessment alone in 88.7% of cases. All methods overpredicted risk, but visual inspection of plots showed the SORT to have the best calibration. The SORT demonstrated the best discrimination of the objective tools (SORT Area Under Receiver Operating Characteristic curve [AUROC] = 0.90, 95% confidence interval [CI]: 0.88–0.92; P-POSSUM = 0.89, 95% CI 0.88–0.91; SRS = 0.85, 95% CI 0.82–0.87). Subjective assessment demonstrated good discrimination (AUROC = 0.89, 95% CI: 0.86–0.91) that was not different from the SORT (p = 0.309). Combining subjective assessment and the SORT improved discrimination (bootstrap optimism-corrected AUROC = 0.92, 95% CI: 0.90–0.94) and demonstrated continuous Net Reclassification Improvement (NRI = 0.13, 95% CI: 0.06–0.20, p < 0.001) compared with subjective assessment alone. Decision-curve analysis (DCA) confirmed the superiority of the SORT over other previously published models, and the SORT–clinical judgement model again performed best overall. Our study is limited by the low mortality rate, by the lack of blinding in the ‘subjective’ risk assessments, and because we only compared the performance of clinical risk scores as opposed to other prediction tools such as exercise testing or frailty assessment.
Conclusions
In this study, we observed that the combination of subjective assessment with a parsimonious risk model improved perioperative risk estimation. This may be of value in helping clinicians allocate finite resources such as critical care and to support patient involvement in clinical decision-making.
Klíčová slova:
Cancer risk factors – Death rates – Forecasting – Instrument calibration – Medical risk factors – Obstetric procedures – Surgical and invasive medical procedures – Vascular surgery
Zdroje
1. Meara JG, Leather AJ, Hagander L, Alkire BC, Alonso N, Ameh EA et al. Global Surgery 2030: evidence and solutions for achieving health, welfare, and economic development. Lancet. 2015;386: 569–624. doi: 10.1016/S0140-6736(15)60160-X 25924834
2. Lees N, Peden CJ, Dhesi J, Quiney N, Lockwood S, Symons NR et al. The High-Risk General Surgical Patient: Raising the Standard. Updated recommendations on the Perioperative Care of the High-Risk General Surgical Patient [Internet]. 2018 [cited 2020 Jun 22]. https://www.rcseng.ac.uk/-/media/files/rcs/news-and-events/media-centre/2018-press-releases-documents/rcs-report-the-highrisk-general-surgical-patient—raising-the-standard—december-2018.pdf
3. Levine GN, O’Gara PT, Beckman JA, Al-Khatib SM, Birtcher KK, Cigarroa JE et al. Recent Innovations, Modifications, and Evolution of ACC/AHA Clinical Practice Guidelines: An Update for Our Constituencies: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;139: e879–e886. doi: 10.1161/CIR.0000000000000651 30892927
4. Mchale JV. Innovation, informed consent, health research and the Supreme Court: Montgomery v Lanarkshire—a brave new world. Health Econ Policy Law. 2017;12: 435–452. doi: 10.1017/S174413311700010X 28446256
5. Biccard BM, Madiba TE, Kluyts HL, Munlemvo DM, Madzimbamuto FD, Basenero A et al. Perioperative patient outcomes in the African Surgical Outcomes Study: a 7-day prospective observational cohort study. Lancet. 2018;391: 1589–1598. doi: 10.1016/S0140-6736(18)30001-1 29306587
6. Ghaferi AA, Birkmeyer JD, Dimick JB. Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg. 2009;250: 1029–1034. doi: 10.1097/sla.0b013e3181bef697 19953723
7. Wong DJN, Harris SK, Moonesinghe SR, SNAP-2: EPICCS Collaborators. Cancelled operations: a 7-day cohort study of planned adult inpatient surgery in 245 UK National Health Service hospitals. Br J Anaesth. 2018;121: 730–738. doi: 10.1016/j.bja.2018.07.002 30236235
8. Weiser TG, Haynes AB, Molina G, Lipsitz SR, Esquivel MM, Uribe-Leitz T et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet. 2015;385(Suppl 2): S11.
9. Moonesinghe SR, Harris S, Mythen MG, Rowan KM, Haddad FS, Emberton M et al. Survival after postoperative morbidity: a longitudinal observational cohort study. Br J Anaesth. 2014;113: 977–984. doi: 10.1093/bja/aeu224 25012586
10. PQIP Project Team. 1st annual report of the Perioperative Quality Improvement Programme. 2018 [cited 2019 December 13]. https://pqip.org.uk/pages/ar2018
11. Abbott TEF, Fowler AJ, Dobbs TD, Harrison EM, Gillies MA, Pearse RM. Frequency of surgical treatment and related hospital procedures in the UK: a national ecological study using hospital episode statistics. Br J Anaesth. 2017;119: 249–257. doi: 10.1093/bja/aex137 28854546
12. Khuri SF, Henderson WG, DePalma RG, Mosca C, Healey NA, Kumbhani DJ. Determinants of long-term survival after major surgery and the adverse effect of postoperative complications. Ann Surg. 2005;242: 326–341. 16135919
13. Toner A, Hamilton M. The long-term effects of postoperative complications. Curr Opin Crit Care. 2013;19: 364–368. doi: 10.1097/MCC.0b013e3283632f77 23817032
14. Partridge JS, Harari D, Dhesi JK. Frailty in the older surgical patient: a review. Age Ageing. 2012;41: 142–147. doi: 10.1093/ageing/afr182 22345294
15. Wijeysundera DN, Pearse RM, Shulman MA, Abbott TEF, Torres E, Ambosta A et al. Assessment of functional capacity before major non-cardiac surgery: an international, prospective cohort study. Lancet. 2018;391: 2631–2640. doi: 10.1016/S0140-6736(18)31131-0 30070222
16. Moonesinghe SR, Mythen MG, Das P, Rowan KM, Grocott MP. Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: qualitative systematic review. Anesthesiology. 2013;119: 959–981. doi: 10.1097/ALN.0b013e3182a4e94d 24195875
17. Peden CJ, Stephens T, Martin G, Kahan BC, Thomson A, Rivett K et al. Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial. Lancet. 2019;393: 2213–2221. doi: 10.1016/S0140-6736(18)32521-2 31030986
18. Moonesinghe SR, Wong DJN, Farmer L, Shawyer R, Myles PS, Harris SK et al. SNAP-2 EPICCS: the second Sprint National Anaesthesia Project-EPIdemiology of Critical Care after Surgery: protocol for an international observational cohort study. BMJ Open. 2017;7: e017690. doi: 10.1136/bmjopen-2017-017690 28882925
19. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61: 344–349. doi: 10.1016/j.jclinepi.2007.11.008 18313558
20. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg. 2015;102: 148–158. doi: 10.1002/bjs.9736 25627261
21. Wong DJN, Popham S, Wilson AM, Barneto LM, Lindsay HA, Farmer L et al. Postoperative critical care and high-acuity care provision in the United Kingdom, Australia, and New Zealand. Br J Anaesth. 2019;122: 460–469. doi: 10.1016/j.bja.2018.12.026 30857602
22. Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity. Br J Surg. 1998;85: 1217–1220. 9752863
23. Sutton R, Bann S, Brooks M, Sarin S. The Surgical Risk Scale as an improved tool for risk-adjusted analysis in comparative surgical audit. Br J Surg. 2002;89: 763–768. doi: 10.1046/j.1365-2168.2002.02080.x 12027988
24. Protopapa KL, Simpson JC, Smith NC, Moonesinghe SR. Development and validation of the Surgical Outcome Risk Tool (SORT). Br J Surg. 2014;101: 1774–1783. doi: 10.1002/bjs.9638 25388883
25. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21: 128–138. doi: 10.1097/EDE.0b013e3181c30fb2 20010215
26. Swets JA. Measuring the Accuracy of Diagnostic Systems. Science. 1988;240: 1285–1293. doi: 10.1126/science.3287615 3287615
27. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44: 837–845. 3203132
28. Pencina MJ, D’Agostino RB, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27: 157–72; discussion 207. doi: 10.1002/sim.2929 17569110
29. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26: 565–574. doi: 10.1177/0272989X06295361 17099194
30. National Institute for Health and Care Excellence. Routine preoperative tests for elective surgery (NICE guideline NG45). 2016 [cited 2019 December 13]. https://www.nice.org.uk/guidance/ng45
31. Graham JW. Analysis of Missing Data. In: Graham JW, ed. Missing Data: Analysis and Design. New York: Springer; 2012. pp. 47–69.
32. Devereaux PJ, Chan MT, Alonso-Coello P, Walsh M, Berwanger O, Villar JC et al. Association between postoperative troponin levels and 30-day mortality among patients undergoing noncardiac surgery. JAMA. 2012;307: 2295–2304. doi: 10.1001/jama.2012.5502 22706835
33. Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217: 833–42.e1. 24055383
34. Chan DXH, Sim YE, Chan YH, Poopalalingam R, Abdullah HR. Development of the Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator for prediction of postsurgical mortality and need for intensive care unit admission risk: a single-center retrospective study. BMJ Open. 2018;8: e019427. doi: 10.1136/bmjopen-2017-019427 29574442
35. Moran J, Wilson F, Guinan E, McCormick P, Hussey J, Moriarty J. Role of cardiopulmonary exercise testing as a risk-assessment method in patients undergoing intra-abdominal surgery: a systematic review. Br J Anaesth. 2016;116: 177–191. doi: 10.1093/bja/aev454 26787788
36. Corey KM, Kashyap S, Lorenzi E, Lagoo-Deenadayalan SA, Heller K, Whalen K et al. Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): A retrospective, single-site study. PLoS Med. 2018;15: e1002701. doi: 10.1371/journal.pmed.1002701 30481172
Článek vyšel v časopise
PLOS Medicine
2020 Číslo 10
- S diagnostikou Parkinsonovy nemoci může nově pomoci AI nástroj pro hodnocení mrkacího reflexu
- Proč při poslechu některé muziky prostě musíme tančit?
- Chůze do schodů pomáhá prodloužit život a vyhnout se srdečním chorobám
- „Jednohubky“ z klinického výzkumu – 2024/44
- Je libo čepici místo mozkového implantátu?
Nejčtenější v tomto čísle
- Association of technologically assisted integrated care with clinical outcomes in type 2 diabetes in Hong Kong using the prospective JADE Program: A retrospective cohort analysis
- Correction: Social distancing to slow the US COVID-19 epidemic: Longitudinal pretest–posttest comparison group study
- The impact of continuous quality improvement on coverage of antenatal HIV care tests in rural South Africa: Results of a stepped-wedge cluster-randomised controlled implementation trial
- Variation in racial/ethnic disparities in COVID-19 mortality by age in the United States: A cross-sectional study
