Data validation is a major challenge for busy trauma registrars and registry managers. What are the most efficient ways to ensure trauma data quality? With this month’s Q&A, the trauma registry experts from Pomphrey Consulting launch a three-article series explaining the basic concepts and strategies of data validation.

In the March column, we will explain how to use software reports to ensure data quality from a structural point of view. Later, we will share strategies for performing “hands on” data validation.

But first, what does the Orange Book require? This month, we clarify what the American College of Surgeons (ACS) does and does not say about data validation.

Q. What does the Orange Book say about data validation?

The Orange Book states that valid data is essential to ensuring the quality and reliability of the trauma registry. According to Chapter 15, trauma centers must establish “a plan for ensuring that the data entered are accurate and reflect the observations made on the patient.”

The related verification criterion applies to all trauma centers, regardless of level: “Strategies for monitoring data validity are essential (CD 15-10).” Failure to meet this criterion is a Type II deficiency.

According to the Orange Book, “If three or fewer Type II deficiencies are present at the time of the site visit and no Type I criteria are cited, a 1-year certificate of verification is issued [emphasis added].” In other words, your trauma center can lose its full 3-year verification if demonstration of your facility’s performance of data validation is not done to satisfaction. That said, while this particular deficiency can contribute to a failed verification, it will not, in and of itself, sink your program.

Q. Does the Orange Book specify how to perform data validation?

No, the ACS does not require trauma centers to use any particular data validation strategy. However, the Orange Book does describe a basic strategy that it regards as acceptable.

According to Chapter 15, “One approach is to re-abstract 5 to 10 percent of patient records. The medical director, trauma program manager, and trauma registrar then can perform a systematic review of the differences to establish levels of inter-rater reliability.”

This suggestion is useful, but it requires some additional exploration. One question is timing. Some registrars believe this passage suggests performing data validation during a single end-of-the-year review. This is not practical. We recommend performing data validation on a monthly basis. Monthly validation makes the entire process more manageable. It also allows team members to catch mistakes early and make corrections before errors persist throughout the year.

The other question is scope. Say your trauma registry collects 500 data elements for every patient. Does that mean you must re-abstract all 500 pieces of data for every record in your validation sample? No, it is sufficient to focus on a small number of key data elements, such as elements that affect risk adjustment and benchmarking. (We’ll talk more about “hands on” data validation techniques in future columns.)

Q. Who is responsible for data validation?

As described in the Orange Book, data validation is a multidisciplinary effort.

• Chapter 15 lists “ongoing data validation” as one of the core tasks of the trauma registrar.
• In addition, ensuring data integrity is part of the trauma program manager’s responsibility to oversee the trauma registry. According to Chapter 5, the TPM’s written job description should include “Supervise collection, coding, scoring, and developing processes for validation of data.”
• And, as noted above, Chapter 15 suggests that the trauma medical director should also take part in data validation by reviewing discrepancies identified in the validation process.

Q. We participate in TQIP, so doesn’t that cover data validation for us?

The Trauma Quality Improvement Program (TQIP) applies validation checks to data submitted by participating centers. However, participation in TQIP does not automatically fulfill data validation requirements.

According to the Orange Book, “While the TQIP of the ACS is an extremely important process that involves extensive external data validation, TQIP participation alone does not necessarily ensure data validity across the entire spectrum of data contained in the hospital trauma registry.”

In general, TQIP provides two forms of data validation:

• The program analyzes submitted data for obvious structural and logical errors. For example, if a trauma center submits 5,000 patients to TQIP and none of them have any comorbid conditions, the program identifies this as a likely instance of missing data.
• TQIP also conducts trauma center site visits. During these visits, TQIP reviewers re-abstract a sample of patient charts and compare them to the submitted data for these patients.

Both services are valuable and can support overall data quality. However, TQIP data checks do not stand by themselves as a complete data validation program. Structural data checks can catch high-level errors, but they may miss serious ongoing problems in chart abstraction and data reporting. And while the site visit is an excellent learning tool, reviewers re-abstract only a small handful of charts. The sample is too small to effectively validate data quality for a trauma registry.

Trauma Registry Q&A is a bimonthly column produced in partnership with Pomphrey Consulting, a comprehensive trauma registry company which provides training and trauma registry management. The American College of Surgeons has recognized our Trauma Registrar Mentorship Program for meeting the trauma registrar course requirement of CD15-7. Our course is revised each year to meet any and all ACS NTDB changes, as well as the demands of the ever-changing world of trauma registries. Our program also includes full trauma registrar training in ICD-10-CM and ICD-10-PCS. To visit our website, click here.