14 Apr Why the $%%#$ does the SRTR keep changing the PSRs?
By: David Axelrod, MD, MBA
The January Program-Specific Reports produced by SRTR included marked changes in the expected death and graft failure rate for many centers, and especially for those with liver transplant programs. Changes in the center’s expected rate impacts the O:E ratio used for center performance assessment. Rapid shifts in the predictive equations can result in programs moving from ‘in compliance’ to ‘out of compliance’ with CMS, UNOS, or private payer Center of Excellence criteria. While these shifts can occur with each cycle, the recent changes are demonstrably larger than what has been seen in the past.
The concern caused by these changes was compounded by the brief release of the 5-tier rating system. The five tier system, although less prominent on the SRTR website, is still available for program and public use (beta.srtr.org) and may return in the future.
Several of our clients have asked: What is the basis of the changes in the equations used by the SRTR, what they can do about it, how do we mitigate its effects, and what will happen going forward. We at XynManagement would like to take a few minutes to explain what you are seeing and will also be happy to answer questions directly.
Why do the expected values change from report to report?
The SRTR prepares new statistical models for transplant risk adjustment every 6 months. In general, these changes are modifications to the weight given to individual factors (e.g. MELD score, age). However, they can also dramatically change by including new variables that were previously excluded from the models, or changing the way a specific factor is included in the model (e.g. changing from a continuous variable for age to age brackets). Among the biggest changes this cycle in the liver equation are the weights for:
- Donor Height
- Recipient Height
- Recipient Weight
- Previous Transplant (reduced)
- ABO Incompatible
- Previous Abdominal Surgery
- Split Liver
The direction and magnitude of the shift in weight given to these variables varies, resulting in differential impacts on center performance which reflect your center’s practice patterns. However, all centers were impacted given the large shifts and the diversity of the factors that were included.
In addition, the new models from the SRTR are based on current national performance. Over time, outcomes have tended to improve nationally. Therefore, the expected rate of graft failure or death will decline if a center is still doing cases with the same characteristics. The focus on early post-transplant outcomes continually raises the bar for center performance. Thus, doing as well as you have always done will not necessarily protect your center if you do not increase your risk.
What tools are available to help you predict these changes?
Our clients can utilize XynPlot to provide real time monitoring of transplant outcomes. If your outcomes are at least at the national average for the last report, it is very unlikely that you will be in regulatory trouble if the models shift. If your XynPlot suggests that your center is at the edge of being flagged for poor performance, it is hard to predict exactly what will happen. For this reason, centers need to proactively monitor their outcomes to address performance changes early and avoid being on the edge.
We feel strongly that the only thing worse than a bad center specific report is a surprise bad center specific report. To this end, we purposely developed the statistical models of XynPlot to be conservative. This means XynPlot is likely to suggest that the center is in trouble, or ‘too close to call’ for a flag from MPSC or CMS, even though it may not be once the new models are released. This statistical conservatism was designed so that transplant centers are always prepared for a bad report, and will be pleasantly surprised if one doesn’t occur.
Monitoring with XynPlot also provides a forward look at potential periods of trouble, well in advance of the actual release of the reports. This allows centers to proactively intervene to change processes, review organ and patient acceptance practices, and manage the message to hospital leadership, patients and payers. For many of our clients, a XynPlot that shows improving outcomes in subsequent reports has been the basis of successful appeals of losses of Centers of Excellence.
XynPlot also adjusts for changes in your practice patterns over time. Thus, a 10% graft failure rate may be okay during one cycle and not okay during the next if the center scales back organ acceptance or patient listing practices.
CUSUM charts provide another potential tool for monitoring performance. While not designed to align with SRTR reporting periods, the CUSUMs provide a glimpse at current performance compared to national average. Again, the key to success is identifying trends in performance early and improving quality in performance. The CUSUM charts are available on the SRTR secure site for key SRTR reported outcomes, and complement the XynPlot and center scorecards.
What should we do given these ongoing changes?
Although multiple changes seem to be occurring simultaneously (regulatory, SRTR reporting, equations and models), we believe that the keys to transplant quality and outcomes remains transparency, systematic review, data integrity, and team empowerment.
- Share the data with your team. The center outcomes (including XynPlot data) should be reviewed at every quality meeting and frequently at organ specific meetings with all team members. People are more easily able to improve in response to feedback and data.
- Systematically review all events, and develop a standardized way to track variances and outcomes. If the XynPlot shows outcomes are starting to fall off track, what has changed? Which population of patients are having poor outcomes in your center? Are you following your processes in a way that ensures good outcomes?
- Create a sustainable process for data capture and reporting to ensure that your tier data is accurate and complete. Missing data is now ASSUMED to reflect the lowest risk of bad outcomes. Therefore, missing data reduces your expected graft failures/deaths. You need to avoid missing or incomplete data through chart reviews, data submission reviews, and the monthly data validation sheet provided by XynManagement. An ongoing evaluation of data entry practices and accuracy ensures the entire team has a deep understanding that UNOS forms are not busy work, they are the key to the center’s very survival.
- Team empowerment. Quality is a team sport, and no center thrives on the basis of any one discipline. Good waitlist practice results in better operative outcomes, but poor discharge care can undo all the pre-transplant work. Building a quality structure will provide the best defense against changes in the risk adjustment models.
- Do not be afraid to take risk. Remember that risk adjustment, though imperfect, still works. If your center performs well with high risk organs, then use them. This only serves to increase your ‘expected’ and can protect you from lower risk graft losses. But, you need to be smart. Social, compliance, and financial risk is poorly captured by the SRTR. Patients that are medically low risk and psychologically high risk can result in major issues for your center.
The program specific reports hold the key to your transplant center’s future. XynManagement stays constantly abreast of PSR changes to help you maintain high rankings and promote good outcomes. We are here to help.
The XynManagement Team.
What do you think?
Join the discussion below.