While the American College of Surgeons requires verified trauma centers to validate registry data, the Orange Book does not specify a particular data validation methodology. One suggestion is to entirely re-abstract 5% to 10% of patient records. But for busy trauma programs with limited resources, this approach can be burdensome.
According to Kathy Cookman, BS, CSTR, CAISS, owner of KJ Trauma Consulting, LLC, many trauma centers can use a streamlined methodology to perform effective data validation. She recommends focusing on the handful of data elements that have the greatest effect on trauma program performance.
Alternative validation method
“An alternative to re-abstracting the entire patient record is to concentrate on just two dozen or so key data fields,” Cookman said. “The idea is to focus on the data points that have the biggest impact on probability of survival, research, outcomes and performance improvement.”
Cookman’s key fields include cause code, AIS value and initial ED vital signs, data elements that capture essential information about the patient’s injury and initial presentation. Fields such as transport mode and surgeon arrival time represent system performance. Other key fields represent key care metrics such as ICU length of stay and complications. (For a full list of recommended data fields, download 29 Key Fields for Trauma Registry Data Validation.)
To perform data validation using the key fields strategy, run a trauma registry report for the targeted time period, requesting only the data fields you are focusing on. If your registry software includes a randomization function, use it to sample 5% to 10% of patient records. Alternatively, export the data to Excel and apply randomization to the spreadsheet. (Many trauma programs also validate data for all top-level activations and patient deaths.)
“The first step is just to look at the data fields logically,” Cookman said. For example, review and compare the cause code, cause E-code, trauma type and injury detail fields. Do they tell a coherent story?
“Make sure the cause code equals the E-code and trauma type, and that they agree with the scenario described in the injury detail field,” Cookman said. “If the cause code is car crash, but the E-code describes a burn, that could need further investigation.”
After reviewing the key fields for internal logic, the next step is to check the targeted data elements against observations. Review patient records to verify that the registry field data are accurate and appropriately coded.
Tailor to local situation
Cookman said the key fields methodology can be a good alternative for large centers with high trauma patient volumes. She cautioned, however, that this method is not appropriate for every hospital.
“If you have experienced registrars and no huge data issues, then looking just at key fields is an appropriate method for data validation,” she said. “However, if you have new staff in the registry or other issues with your data, then the full-blown approach is better.”
Cookman said trauma programs should tailor the key fields strategy to their unique situation and needs. “It makes sense to include your program’s audit filters in the data fields you look at,” she said. “Some centers also validate procedure information, mainly intubation, CPR, chest tube insertion and any operative procedures — the big ticket items that have a major impact on outcomes.”
She also recommends that participants in the ACS Trauma Quality Improvement Program include TQIP Process Measures as key fields for validation. (The Orange Book clarifies that while TQIP provides “extensive external data validation,” verified trauma centers still need to perform ongoing review and evaluation of local registry data.)
3 ways to get the most out of data validation
No matter what data validation methodology you use, make sure that the process ties into performance improvement. Cookman recommends three practices:
1. Record findings in an audit filter.
“A lot of trauma programs are not recording their data validation findings,” Cookman said. “Others are, but they record the findings in the minutes of the trauma PI committee.” Cookman recommends recording data validation findings within the trauma registry itself, via an audit filter in the PI section. “That way you can track and trend data validation findings and report on them.”
2. Make the process educational.
In some programs, registrars get very little feedback from the data validation process. “The feedback from the lead registrar might be ‘wrong vitals’ and that’s pretty much all they say,” Cookman noted. “It makes more sense to use the process to educate registrars, to make sure they understand the big picture. ‘Here are the vitals, here’s why they are important, and this is how one mistake can impact what we are doing.’ A registrar who understands the ‘why’ part will be much less likely to make the same mistake a second time.”
3. Use findings to build a registry handbook.
“One thing we always do with our clients is create a trauma registry handbook,” Cookman said. “The handbook will have data definitions along with screenshots that show where to capture specific data elements from the records.” When errors are uncovered during data validation, staff should update the handbook to clarify correct practice. “It comes back to education. Maintaining a trauma registry handbook helps registrars learn from their mistakes and continuously improve.”