The Cribari Grid has long been the standard tool for measuring undertriage and overtriage rates within a trauma system. But while the grid is relatively simple, using it to monitor and improve triage performance is a challenge. According to David Kashmer, MD, chair of surgery at Signature Healthcare in Brockton, Massachusetts, using triage rates incorrectly can actually lead to lower quality care.
The Cribari Grid uses trauma registry data to categorize patients by injury score and activation type. Simple calculations yield the percentage of severely injured patients who did not receive a trauma activation (undertriage) and the percentage of less injured patients who did receive an activation (overtriage).
“The thing that makes the Cribari Grid such a useful tool is that it gives us a simple way to answer the question, How often did we get it right?” Dr. Kashmer said. “The problem is what you do with that information.”
Many trauma programs do not take a true system approach to monitoring triage rates, according to Dr. Kashmer. “Even though they might calculate their under- and overtriage rates regularly, they still monitor triage on a case-by-case basis.”
No compass
To illustrate the point, he described a recent experience at a large Level II trauma center. “The trauma team became very focused on an elderly patient who met the activation criteria but did not receive a trauma activation,” Dr. Kashmer said. “Based on that one case, they made changes to their triage system with the goal of reducing undertriage.”
Cribari calculations later showed that overtriage went way up, as expected. However, undertriage increased as well. “The system became unable to get it right,” Dr. Kashmer said.
Part of the problem was that the system’s leaders had created a very complicated triage protocol. But the main issue was that the system had no compass to guide process change.
“When you are monitoring triage on a case-by-case basis, you are essentially waiting for problems, and that plays to extremes,” Dr. Kashmer said. In addition, case-by-case monitoring makes it impossible to link system changes to outcomes. “You make a change to your triage system and triage rates may improve or they may get worse, but you will never know whether the system changes were the cause.”
Mark the finish line
The solution, according to Dr. Kashmer, is statistical process control — a group of methodologies for improving quality by reducing variation within a process. Rigorous statistical analysis ensures that data can be used to make valid conclusions about what is working or not working within a system.
“When working with triage rates, you need to make sure you have a big enough sample size and you need to do significance testing,” Dr. Kashmer said. He provided the following formula:
(p)(1-p)(2/delta)^2
P is the probability that an event will occur. In this instance, P is the system’s current undertriage rate. Delta is the percentage change in undertriage that you want to be able to detect. The 2 in the equation is the approximate standard score (or “z-score”) at the 95% level of confidence. (Note: “^2” means “squared.”)
“Basically, you plug in your system’s current undertriage rate and the smallest change in the undertriage rate that you want to be able to detect,” Dr. Kashmer said.
Say, for example, that a system’s undertriage rate is about 4%, and system leaders want to be able to detect a 2% change in that rate, either better or worse. Their sample size calculation would be (0.04)(0.96)(2/0.02)^2. The result is 384.
“The equation tells you how large a sample of discrete data you’d need to detect that size change at the 95% level of confidence. So, here, you’d need 384 trauma patient activations to detect a 2% change. Now we know how much data we need to collect before checking to see if our changes worked. We also know how long we’ll be collecting data before we disturb the process again. If we need 384 patients and we get about 125 in a month, our data collection will be approximately 3 months.”
True system perspective
Statistical process control allows trauma leaders to use the Cribari Grid effectively. “Say your Cribari calculation shows that your undertriage rate is x. You can make changes to your system, collect data, and then run statistical tests to determine when you have enough patients under the new system to show a statistically significant change. You can rigorously show whether undertriage got better, or was unchanged, or became worse.”
One benefit of statistical process control is that it gives trauma teams a true system perspective. “When you are focused on individual cases, the discussion becomes, ‘Dr. Smith did something wrong.’ But when you are disciplined about data, it puts everyone on the same page,” Dr. Kashmer said.
“You can look at the numbers and say, ‘This is our performance as a system, and making these changes has resulted in this outcome.’ It keeps your team together while at the same time leading to real improvement.”