Do you have a “gut feeling” about what is causing performance problems in your trauma center? Most people start the PI process with a strong suspicion about which factors are leading to a poor outcome. The problem is that reliance on intuition can prevent a trauma team from making a full and careful evaluation of their system’s shortfalls.
According to David Kashmer, MD, chair of surgery at Signature Healthcare in Brockton, Massachusetts, two well-known quality tools are the key to moving beyond gut feeling in trauma PI. He recently described how these tools helped solve a problem that affects many trauma programs.
“A few years ago, I worked with a trauma center that was having a big problem with diversion,” Dr. Kashmer said. “The hospital had logged more than 200 hours of diversion over a brief time period. Diversion is a complex issue, and it was challenging just to get everyone on the same page.”
The team’s first response was to create an elaborate diversion protocol. “The algorithm specified everyone who needed to be called before the hospital could go on diversion,” Dr. Kashmer said. “But the protocol was so complex, it would take hours to complete. By the time you had all the approvals, the situation on the ground was often completely different.”
At that point, the team committed to using data to understand the problem and find an effective solution. Their approach included two steps:
Step 1: Use an Ishikawa diagram to identify possible causes
The team convened a meeting of all stakeholders to discuss the factors that could be driving diversion at the hospital. Participants used the Ishikawa fishbone technique to identify the full range of possible causes. This method helps groups consider causes in seven different categories—method, machine, materials, man (people), management and mother nature.
“We tried to identify all the things we thought might correlate with being on diversion,” Dr. Kashmer said. “In the end we came up with about a dozen factors.” Potential causes included:
- Time of day
- Day of week (weekend versus weekday)
- ED physician on duty
- Availability of ED beds
- Number of available ICU beds
- Availability of inpatient floor beds
- ICU census
- Number of ORs available
- Weather
While the fishbone exercise is an important first step, it does have limitations. “The challenge of a fishbone diagram is keeping the results in perspective,” Dr. Kashmer said. “We feel good about them, but can we ever regard them as more than just a team’s opinions about the system?”
Step 2: Perform multiple regression analysis
The key to moving beyond opinion is to perform a multiple regression analysis. Multiple regression allows you to determine which variables (time of day, day of week, ED physician, etc.) are correlated with a given outcome (in this case, diversion status).
“We began by getting data on our diversion time, then we used multiple regression to determine which, if any, of the possible causes identified in the fishbone exercise correlated with being on diversion,” Dr. Kashmer said. To use multiple regression effectively, pay attention to three points:
Overall fitness of your model. “The number to focus on is called the r-squared adjusted value, or RSA,” Dr. Kashmer said. “It is the percentage of variability in Y — the outcome — that is accounted for by the X’s included in your model.” For example, if the RSA of a multiple regression is 0.65, that means 65% of variation in the outcome is due to the model’s independent variables. “Ideally, you want an RSA of 0.80, meaning 80% of the variability in your Y is explained by the X’s in your model.”
Statistical significance. Another important element of multiple regression analysis is determining the p value associated with each independent variable. “Here, you are determining whether any of the X’s affect your Y in a statistically significant manner,” Dr. Kashmer said.
Controllable versus non-controllable variables. As a final layer of analysis, label each potential factor as either “controllable” or “noise” and rerun the analysis. “We remove those factors that we cannot control and run the model again to determine what portion of the outcome is in our control,” he said. “In other words, we want to find out what portion of the outcome we can control through our choices.”
A good starting point
According to Dr. Kashmer, the diversion model developed by the trauma team produced an RSA value of less than 0.80, so the variables explained less than 80% of the variation in the center’s diversion. However, the data did identify one variable that had a statistically significant effect on diversion — number of ICU beds available. “The data showed that this was the single thing that seemed to matter. When we didn’t have ICU beds, we were diverting and our time on diversion was longer, ” Dr. Kashmer said. “Our model wasn’t perfect, but this gave us a starting point to correct the issue.”
Based on this data, the hospital flexed up nursing staff in the ICU, increasing the unit’s ability to accept patients. Improvement was dramatic. According to Dr. Kashmer, the hospital did not go on diversion again during the next 24 months. “When we decided to use rigorous data-driven techniques, it took us about one month to sort out the issues and then another month to implement the fix,” he said.
Three cautions
Dr. Kashmer noted three issues that trauma teams should pay attention to when using fishbone diagramming and multiple regression:
1. You may need to perform a special data collection. “What is measured is managed, but what matters is not always measured,” Dr. Kashmer said. “In our case, factors identified during the fishbone exercise like diversion time and ‘sunny weather versus rainy weather’ were not in our registry. We had to track them in order to make our model.”
2. Consider getting help with the number crunching. “You can do a multiple regression with software like Mini Tab, but there are still some very easy ways to mess it up,” Dr. Kashmer said. “For most people, it makes sense to get some statistical help.” He encourages trauma teams to make use of hospital statisticians or other research support resources.
3. Remember to focus on your problems, not someone else’s. “The wrong takeaway for anyone reading this is that the key to reducing diversion is to add ICU beds,” Dr. Kashmer said. “That was the problem in a particular system, but it may not be the same for everybody.” For other trauma centers, the root cause of a high diversion rate might be patient volume or surgeon response times or several other factors. “The value of using a statistically rigorous approach is that it tell you what the problem is at your institution.”
Did you like this article? Our free monthly newsletter includes regular features on trauma quality and performance improvement. Click here to subscribe.