Does the choice of risk-adjustment model influence the outcome of surgeon-specific mortality analysis? A retrospective analysis of 14 637 patients under 31 surgeons

Published on Aug 1, 2008in Heart5.213
· DOI :10.1136/HRT.2006.110478
Stuart W Grant22
Estimated H-index: 22
Antony D. Grayson34
Estimated H-index: 34
+ 5 AuthorsBen Bridgewater31
Estimated H-index: 31
Objectives: To compare implications of using the logistic EuroSCORE and a locally derived model when analysing individual surgeon mortality outcomes. Design: Retrospective analysis of prospectively collected data. Setting: All NHS hospitals undertaking adult cardiac surgery in northwest England. Patients: 14 637 consecutive patients, April 2002 to March 2005. Main outcome measures: We have compared the predictive ability of the logistic EuroSCORE (uncalibrated), the logistic EuroSCORE calibrated for contemporary performance and a locally derived logistic regression model. We have used each to create risk-adjusted individual surgeon mortality funnel plots to demonstrate high mortality outcomes. Results: There were 458 (3.1%) deaths. The expected mortality and receiver operating characteristic (ROC) curve values were: uncalibrated EuroSCORE −5.8% and 0.80, calibrated EuroSCORE −3.1% and 0.80, locally derived model −3.1% and 0.82. The uncalibrated EuroSCORE plot showed one surgeon to have mortality above the northwest average, and no surgeon above the 95% control limit (CL). The calibrated EuroSCORE plot and the local model showed little change in surgeon ranking, but significant differences in identifying high mortality outcomes. Two of three surgeons above the 95% CL using the calibrated EuroSCORE revert to acceptable outcomes when the local model is applied but the finding is critically dependent on the calibration coefficient. Conclusions: The uncalibrated EuroSCORE significantly overpredicted mortality and is not recommended. Instead, the EuroSCORE should be calibrated for contemporary performance. The differences demonstrated in defining high mortality outcomes when using a model built for purpose suggests that the choice of risk model is important when analysing surgeon mortality outcomes.
📖 Papers frequently viewed together
968 Citations
2,838 Citations
1,167 Citations
#1Ben Bridgewater (University of Manchester)H-Index: 31
#2Antony D. GraysonH-Index: 34
Last. Bruce E. KeoghH-Index: 33
view all 9 authors...
Objectives: To study changes in coronary artery surgery practice in the years spanning publication of cardiac surgery mortality data in the UK. Methods: A retrospective analysis of prospectively collected data from all National Health Service centres undertaking adult cardiac surgery in northwest England was carried out. Patients undergoing coronary artery surgery for the first time between April 1997 and March 2005 were included. Changes in observed, predicted and risk adjusted mortality (EuroS...
133 CitationsSource
#1Farah Bhatti (Manchester Royal Infirmary)H-Index: 2
#2Antony D. GraysonH-Index: 34
Last. Ben Bridgewater (University of Manchester)H-Index: 31
view all 7 authors...
Objectives: To study the ability of the logistic EuroSCORE to predict operative risk in contemporary cardiac surgery. Design: Retrospective analysis of prospectively collected data. Setting: All National Health Service centres undertaking adult cardiac surgery in northwest England. Patients: All patients undergoing cardiac surgery between April 2002 and March 2004. Main outcome measures: The predictive ability of the logistic EuroSCORE was assessed by analysing how well it discriminates between ...
131 CitationsSource
#1Cheng-Hon Yap (St. Vincent's Health System)H-Index: 16
#2Morteza Mohajeri (St. Vincent's Health System)H-Index: 1
Last. Michael Yii (St. Vincent's Health System)H-Index: 21
view all 6 authors...
Background: The purpose of the present paper was to assess the performance of the European system for cardiac operative risk evaluation (EuroSCORE) model in an Australian adult cardiac surgical population. Methods: The additive and logistic EuroSCORE models were retrospectively applied to predict operative mortality in 2106 consecutive patients undergoing cardiac surgery at St Vincent's Hospital, Melbourne between June 2001 and August 2003, and at Geelong Hospital between June 2001 and April 200...
29 CitationsSource
Public reporting of hospital outcomes was pioneered by New York State, with coronary bypass graft surgery (CABG) statistics available since 1989 and coronary angioplasty (PCI) since 1994. The public release of individual physician data after 1991, subsequent to a lawsuit filed under the Freedom of
25 CitationsSource
‘Funnel plots’ are recommended as a graphical aid for institutional comparisons, in which an estimate of an underlying quantity is plotted against an interpretable measure of its precision. ‘Control limits’ form a funnel around the target outcome, in a close analogy to standard Shewhart control charts. Examples are given for comparing proportions and changes in rates, assessing association between outcome and volume of cases, and dealing with over-dispersion due to unmeasured risk factors. We co...
618 CitationsSource
Health care report cards publicly report information about physician, hospital, and health plan quality in an attempt to improve that quality. Reporting quality information publicly is presumed to motivate quality improvement through 2 main mechanisms. First, public quality information allows patients, referring physicians, and health care purchasers to preferentially select high-quality physicians. Second, public report cards may motivate physicians to compete on quality and, by providing feedb...
512 CitationsSource
Objectives To present named surgeon mortality for isolated first time coronary artery surgery and aortic valve surgery. Design Retrospective analysis of prospectively collected data. Setting All NHS hospitals undertaking adult cardiac surgery in north west England. Participants 25 consultant surgeons carrying out coronary artery surgery and aortic valve replacement between April 2001 and March 2004. Main outcome measures Mortality for both operations according to surgeon. EuroSCORE to stratify p...
63 CitationsSource
#1Craig R. Narins (UR: University of Rochester)H-Index: 19
#2Ann DozierH-Index: 22
Last. Wojciech ZarebaH-Index: 112
view all 4 authors...
Background Public disclosure of physician-specific performance data is becoming increasingly common. However, the influence that public reporting of outcome data has on the delivery of care by physicians who are being assessed is not well understood. Methods Since 1994, the New York State Department of Health has collected and periodically published observed and risk-adjusted patient mortality rates for all interventional cardiologists practicing coronary angioplasty in the state. To assess the ...
173 CitationsSource
OBJECTIVE: To study the use of the additive and logistic European System for Cardiac Operative Risk Evaluation (EuroSCORE) to predict mortality following adult combined coronary artery bypass grafting (CABG) and valve surgery. METHODS: Data were collected prospectively, from all four centres providing adult cardiac surgery in the north west of England, on 1769 consecutive patients undergoing combined CABG and valve surgery between April 1997 and March 2002. Observed in-hospital mortality was com...
48 CitationsSource
#1A. Gogbashian (RCS: Royal College of Surgeons of England)H-Index: 2
#2A. Sedrakyan (RCS: Royal College of Surgeons of England)H-Index: 2
Last. Tom Treasure (St Thomas' Hospital)H-Index: 73
view all 3 authors...
Summary The validity of the cardiac surgical scoring system, EuroSCORE, has been assessed by several individual cardiac centres within and outside Europe. We chose to assess the overall international performance by systematic review of peer-reviewed literature. There were six studies meeting our criteria for assessment. Internationally, the evidence is highly suggestive that additive EuroSCORE performance generally over-estimates mortality at lower EuroSCOREs (EuroSCORE # 6) and under-estimates ...
199 CitationsSource
Cited By29
#1John RigbyH-Index: 10
#2Jillian YeowH-Index: 6
Last. Jose Pineda Mendoza (Universidad Privada San Juan Bautista)
view all 0 authors...
#1Shubhra Sinha (UoB: University of Bristol)H-Index: 3
#2Arnaldo Dimagli (UoB: University of Bristol)H-Index: 4
Last. Umberto Benedetto (UoB: University of Bristol)H-Index: 36
view all 8 authors...
OBJECTIVES The most used mortality risk prediction models in cardiac surgery are the European System for Cardiac Operative Risk Evaluation (ES) and Society of Thoracic Surgeons (STS) score. There is no agreement on which score should be considered more accurate nor which score should be utilized in each population subgroup. We sought to provide a thorough quantitative assessment of these 2 models. METHODS We performed a systematic literature review and captured information on discrimination, as ...
#1Ilona W. M. Verburg (PHRI: Public Health Research Institute)H-Index: 7
#2Rebecca Holman (PHRI: Public Health Research Institute)H-Index: 17
Last. Ameen Abu-Hanna (PHRI: Public Health Research Institute)H-Index: 5
view all 5 authors...
Funnel plots are graphical tools to assess and compare clinical performance of a group of care professionals or care institutions on a quality indicator against a benchmark. Incorrect construction of funnel plots may lead to erroneous assessment and incorrect decisions potentially with severe consequences. We provide workflow-based guidance for data analysts on constructing funnel plots for the evaluation of binary quality indicators, expressed as proportions, risk-adjusted rates or standardised...
18 CitationsSource
#1Samuel H. Howitt (University of Manchester)H-Index: 4
#2Camila C. S. Caiado (Durham University)H-Index: 9
Last. Stuart W Grant (University of Manchester)H-Index: 22
view all 7 authors...
Background Several cardiac surgery risk prediction models based on postoperative data have been developed. However, unlike preoperative cardiac surgery risk prediction models, postoperative models are rarely externally validated or utilized by clinicians. The objective of this study was to externally validate three postoperative risk prediction models for intensive care unit (ICU) mortality after cardiac surgery. Methods The logistic Cardiac Surgery Scores (logCASUS), Rapid Clinical Evaluation (...
7 CitationsSource
Background: The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy. Objectives: To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care. Design: Two separate but related realist sy...
100 CitationsSource
#1Katja Behrendt (Lond: University of London)H-Index: 1
#2Oliver Groene (Lond: University of London)H-Index: 27
Public reporting of surgeon outcomes has become a key strategy in the English NHS to ensure accountability and improve the quality of care. Much of the evidence that supported the design of the strategy originates from the USA. This report aims to assess how the evidence on public reporting could be harnessed for cross-country translation of this health system strategy; in particular, to gauge the expected results of the UK surgeon outcome initiative and to propose criteria that elucidate that p...
13 CitationsSource
#1Graeme L. Hickey (University of Manchester)H-Index: 22
#2Stuart W Grant (UCL: University College London)H-Index: 22
Last. Ben Bridgewater (University of Manchester)H-Index: 31
view all 5 authors...
#1Silvia Saffioti (UCSC: Catholic University of the Sacred Heart)H-Index: 5
#2Francesco BurzottaH-Index: 63
Last. Filippo CreaH-Index: 103
view all 7 authors...
: Risk stratification of patients undergoing open-heart surgery or percutaneous interventions may help physicians select the best individual management of patients with advanced heart disease.European System for Cardiac Operative Risk Evaluation (EuroSCORE) is an easy-to-calculate, clinical scoring system which has been developed for coronary surgery and applied in various cardiological settings. Recently, the EuroSCORE II model has been launched with the aim of improving the stratification perf...
5 CitationsSource
#1Stuart W Grant (UCL: University College London)H-Index: 22
#2Graeme L. Hickey (UCL: University College London)H-Index: 22
Last. Charles McCollum (MAHSC: Manchester Academic Health Science Centre)H-Index: 73
view all 4 authors...
Objective/background A number of contemporary risk prediction models for mortality following elective abdominal aortic aneurysm (AAA) repair have been developed. Before a model is used either in clinical practice or to risk-adjust surgical outcome data it is important that its performance is assessed in external validation studies. Methods The British Aneurysm Repair (BAR) score, Medicare, and Vascular Governance North West (VGNW) models were validated using an independent prospectively collecte...
21 CitationsSource