Since it was commissioned in 1993, cardiothoracic surgery at The James Cook University Hospital has continuously monitored clinical performance, both in collaboration with the Society of Cardiothoracic Surgeons of Great Britain and Ireland, as well as through internal trust clinical governance procedures.
This work involves clinical audit, monitoring routinely collected hospital statistics and utilising specialist clinical databases.
Cardiothoracic services has its own governance team, dedicated to constantly monitoring clinical outcomes and validating the data used for such monitoring.
We have a clear policy for performance monitoring, outlined in these web pages.
The information on these pages shows patient outcomes following cardiac surgical treatment as a comparison of actual against predicted outcomes.
Predicted outcomes are determined from risk scoring systems. These are models, such as the Euroscore, which can calculate the predicted risk of a particular procedure based on a number of variables, such as age, kidney function for any individual patient.
They provide a standard against which we can compare our own results.
Points to consider when looking at these results
- Some patients are at greater risk of dying than others. Surgeons who operate on such patients would be expected to have a higher mortality rate than those that don’t operate on such cases.
- There is a natural variation in mortality rates from year to year, so the mortality rate of an individual surgeon in one particular year does not accurately reflect their overall performance.
- The statistical uncertainty in the mortality rate of an individual surgeon depends on how many operations they have performed. The more operations performed, the more confident we can be that the figures are an accurate reflection of the ‘true’ mortality rate. Judging performance on small numbers of operations will lead to incorrect conclusions.
- Mortality rates depend on more than the skill of the individual surgeon. Our surgeons work as part of a large team; all the members of the team can influence patient outcome. These include anaesthetists, perfusionists, physiotherapists, nurses, junior medical staff and pharmacists. The hospital facilities and environment also influence clinical outcome.
Validation of data
To ensure accurate data is used for monitoring, the unit has a team of clinically qualified staff dedicated to the collection and validation of data for all patients receiving cardiac surgery.
They report both on clinical outcomes and the quality of the data collected.
Validation checks include random case note selection, to check the accuracy of the data entered, and processes to ensure that the data is complete.
Since 2003, we have reported data nationally to the Central Cardiac Audit Database (CCAD).
All data is electronically shared with Society of Cardiothoracic Surgeons of Great Britain and Ireland (SCTS) for inclusion in its annual publication of audit data for cardiac surgery.
To enable care to be monitored reliably through time, three main groups of patients who receive cardiac surgery are reported separately so that rare, high-risk or unusual cases do not introduce ‘false-alarms’ into the monitoring process.
Thus three data sets are reported here.
The first reports on patients who receive coronary artery bypass grafts (CABG) for the first-time and without another concurrent cardiac procedure (such as a repair to a heart valve, for example).
The second reports on patients who receive aortic valve replacement surgery (AVR) for the first-time and without another concurrent cardiac procedure.
The third reports on all cardiac surgical procedures performed.
This includes first and subsequent procedures performed. It will also include surgery where concurrent procedures are perfomed.
The data is updated quarterly and shows in-hospital mortality. We have included a rolling timescale of three yearly data up to the point where the data has been accurately validated.
Our processes of data collection, validation, analysis and governance were externally reviewed in July 2005 under the auspices of the SCTS, CCAD, and the Healthcare Commission.
The full report has been received by the Trust and is available for inspection. We were proud that it concluded by stating that “…this is an absolutely outstanding cardiac surgical audit programme which reflects the energy, enthusiasm, and innovation of a synergistic and forward-looking team.
The visitors were particularly impressed by the importance placed on the accuracy and validation of data.”
Performance monitoring
We have a number of procedures in place that enable ongoing performance monitoring.
The governance team constantly monitor surgical mortality, as well as producing reports and validating the data.
We have clearly defined, and highly rigorous, levels of ‘acceptable performance’.
If an individual falls outside these limits, it automatically triggers a chain of events that includes a full re-assessment of results, identification of any problems and, after determining measures to correct any problem, a clear process of reassessment.
These processes also involve clinicians outside the surgical directorate, the trust’s own clinical governance team and, if necessary, clinicians from other organisations.
Setting ourselves such high standards and having clearly defined performance monitoring protocols, enables early detection of any potential underperformance and gives us confidence in the level of care we provide.
Results
We believe that openness and transparency can bring benefits for all, but that there are new dangers involved in sharing this data widely.
Please keep this uppermost in your mind as you look through the information on these pages. Debate exists in the literature about the best way of displaying this kind of data.
As part of our ongoing commitment to research and audit, the unit is currently testing a number of other methods for graphically representing performance.
We believe, as do others, that funnel plots may be the best way forward (Spiegelhalter 2005).
The data is therefore represented below in this format. In funnel plots, the red lines (known as upper and lower control limits) are calculated around the mean predicted mortality for the whole unit, instead of separately for the observed mortality for each team.
Each team is then displayed by plotting the observed mortality rate against the volume of cases.
This shows that all surgeons are performing better than the average prediction, and that there are none that are close to the upper, curved limit, which would raise concerns about a significant difference between surgical teams’ performance if crossed.
Graph one
Funnel charts of CABG actual and predicted mortality with three sigma control limits using Logistic Euroscore II.
Graph two
Funnel charts of AVR actual and predicted mortality with three sigma control limits using Logistic Euroscore II.
Graph three
Funnel charts of all cardiac surgical procedures performed using Logistic Euroscore II.
What do these graphs demonstrate?
These charts demonstrate that for patients undergoing cardiac surgery, our surgeons as a team perform better than Euroscore predictions.
There is no significant difference between the results of the individual surgeons.