The ACS Committee on Trauma requires all verified trauma centers to use a risk-adjusted benchmarking system to assess patient outcomes (CD 15–5). In addition, benchmarking is a required element of trauma PI loop closure (CD 16–2).
In my experience, the word “benchmarking” drives fear into many a heart. However, benchmarking is simply a process in which you evaluate something by comparing it with a standard.
The ACS Trauma Quality Improvement Program (TQIP) provides one opportunity for participating trauma centers to benchmark their performance against national standards. However, there are other ways to benchmark patient outcomes.
In this article, I explain how to benchmark trauma center outcomes using M and Z scores (also known as M and Z statistics). Even if your hospital is a TQIP center, M and Z scores can be very useful. They provide another tool for identifying poor outcomes and improving trauma care.
Trauma M score: How well your patient case mix matches the comparison group
The first step is to determine your trauma center’s M score. The M score tells you whether your case mix of patients is similar to a national comparison group. If your case mix is similar, you can make a valid comparison. If it isn’t, you can’t.
The national comparison group is the group of trauma patients that make up the Major Trauma Outcome Study (MTOS). The MTOS has its drawbacks (mainly the age of the study), but it does provide a very large benchmark group for comparing trauma patient outcomes — more than 80,000 trauma patients treated at 139 hospitals in North America.
There are four steps to calculating the M score:
1. Determine what fraction of your patients fall into each of six probability of survival (Ps) increments. Here is a theoretical example of a trauma center’s patient mix grouped by survival probability:
2. Compare those fractions to the corresponding fractions of the MTOS group. Here is the same scenario, with the addition of MTOS patient mix fractions:
3. Quantify the difference between the two fraction sets (the M Score) using the following formula:
M = s1 + s2 + s3 + s4 + s5 + s6
In this formula, “s” is the smaller of the two values in each Ps increment. For example, s1 is the smaller of the two values in the first Ps increment, s2 is the smaller of the two values in the second Ps increment, and so on.
Using the values in the scenario above, the formula is:
M = s1 + s2 + s3 + s4 + s5 + s6
M = 0.0828 + 0.045 + 0.044 + 0.000 + 0.017 + 0.010
M = 0.944
4. Determine whether your patient mix is “similar” or “dissimilar” to the MTOS group.
In general, the closer your trauma center’s M score is to 1.0, the more your patient population is a “match” to the MTOS study group. Specifically:
- If your M score is between 0.88 and 1.0, your patient case mix is considered similar to the MTOS case mix.
- If your M score is between 0.0 and 0.87, your patient case mix is considered dissimilar to the MTOS case mix.
In the theoretical example above, the M score is 0.944, representing an excellent “match” between the trauma center’s patient group and the MTOS study group. The hospital’s trauma patient population is statistically similar to the MTOS benchmark population.
If your hospital’s M score is similar to the MTOS benchmark, the next step is to calculate your Z score.
Trauma Z score: How well your patient outcomes match the comparison group
The Z score compares the outcomes of two subsets of a patient population. In this case, the Z score quantifies the difference between the actual number of deaths (or survivals) in your center’s patient population and the predicted number of deaths (or survivals) based on the MTOS baseline.
The good news is that most trauma registry software packages can produce a Z statistic report. The only thing trauma leaders need to do is understand how their software performs this calculation. There are two possibilities:
- Your software calculates the Z score based on deaths. In this case, the desired result is a negative number. This implies that your center’s actual deaths are less than the predicted number of deaths.
- Your software calculates the Z score based on survivals. In this case, the desired result is a positive number — implying that more of your center’s patients survived than predicted by the MTOS baseline.
To recap, the Z score can be calculated based on either deaths of survivals. The sign (+ or -) of Z changes, but the absolute value does not.
You still need to determine whether any variation is significant. Here is helpful information from an article in Annals of Surgery: “If z is more negative than −1.96, significantly more patients died than were predicted. If z exceeds +1.96, significantly more patients survived than predicted[.]”
If your trauma center’s Z score indicates significantly worse outcomes, address the problem in your performance improvement and patient safety (PIPS) process. Review unexpected outcomes to identify any individual or system-wide failures and develop recommendations for future management or guideline revisions. The Z score itself will provide a sound benchmarking method, thus closing the loop for your PIPS committee.
What do you do if your M score is dissimilar to the MTOS group?
If the severity of your patient group does not match the MTOS group, you cannot use MTOS benchmarks to evaluate your trauma center’s performance. However, there are other options.
In the next article from Pomphrey Consulting, I will explain how to use W scores as a versatile tool for benchmarking trauma center performance.