Benchmarking of survival outcomes following haematopoietic stem cell transplantation: A review of existing processes and the introduction of an international system from the European Society for Blood and Marrow Transplantation (EBMT) and the Joint Accreditation Committee of ISCT and EBMT (JACIE)

Abstract

In many healthcare settings, benchmarking for complex procedures has become a mandatory requirement by competent authorities, regulators, payers and patients to assure clinical performance, cost-effectiveness and safe care of patients. In several countries inside and outside Europe, benchmarking systems have been established for haematopoietic stem cell transplantation (HSCT), but access is not universal. As benchmarking is now integrated into the FACT-JACIE standards, the EBMT and JACIE established a Clinical Outcomes Group (COG) to develop and introduce a universal system accessible across EBMT members. Established systems from seven European countries (United Kingdom, Italy, Belgium, France, Germany, Spain, Switzerland), USA and Australia were appraised, revealing similarities in process, but wide variations in selection criteria and statistical methods. In tandem, the COG developed the first phase of a bespoke risk-adapted international benchmarking model for one-year survival following allogeneic and autologous HSCT based on current capabilities within the EBMT registry core dataset. Data completeness, which has a critical impact on validity of centre comparisons, is also assessed. Ongoing development will include further scientific validation of the model, incorporation of further variables (when appropriate) alongside implementation of systems for clinically meaningful interpretation and governance aiming to maximise acceptance to centres, clinicians, payers and patients across EBMT.

Fig. 1

Timeline for projected development of clinical outcomes benchmarking process and scoping of established benchmarking systems

Fig. 2

Funnel plot for 1-year mortality following allogeneic transplantation comparing observed over expected mortality adjusted for case mix and centre follow-up. Results are highly affected by quality of follow-up over the period. *Adjusted for case mix and centre follow up

Fig. 3

Funnel plot for 1-year loss to follow-up for allogeneic transplants. *Adjusted for patient and centre-specific mortality

Fig. 4

Funnel plot for 1-year mortality following autologous transplantation comparing observed over expected mortality adjusted for case mix and centre follow-up. Results are highly affected by quality of follow-up over the period. *Adjusted for case mix and centre follow up

Fig. 5

Funnel plot for 1-year loss to follow-up for autologous transplants. *Adjusted for patient and centre-specific mortality