Ongoing data validation is essential to ensuring the quality of a trauma registry. In January’s Q&A column, the trauma registry experts at Pomphrey Consulting clarified what the American College of Surgeons expects regarding data validation. This month, they explain how to use registry software reports to ensure data quality from a structural point of view.

Q. Which reports should be part of our data validation plan?

Trauma registrars, data analysts and program managers should focus on two kinds of reports — National Trauma Data Bank (NTDB) Validator reports and “data cleaning” reports.

Validator reports use NTDB edit checks that have been built into your registry software. Data cleaning reports are other reports you can run using your software’s reporting functionality. Both report types help you perform data validation by flagging errors and common data quality problems.

Q. How do you use NTDB Validator reports in data validation?

First, run the NTDB Validator report and save it. Then, depending on your software, you can either open the error log in Excel or upload the XML file at the NTDB Data Center and click “Submit”. The NTDB system will generate a report that lists your errors. Your data will not actually be submitted to the NTDB until you “Accept” the file. Either method allows you to see your errors. (Just remember that the NTDB Validator report only identifies schematic errors.)

Next, investigate and resolve the errors listed in the Validator report. The report flags four different categories of errors:

• Level 1 flags are format or schema errors. These errors often involve values that are outside of the valid range — for example, a patient age value that is greater than 120 years.
• Level 2 flags identify errors in data fields that help determine whether the patient file meets the NTDB inclusion criteria or that are required for critical analyses. For example, the NTDB will not accept patient files if the race, sex or ethnicity field is blank.
• Level 3 flags identify “major logic” errors. For example, if Report of Physical Abuse is marked “Yes,” then the Investigation of Physical Abuse field cannot be marked “Not Applicable.”
• Level 4 flags identify “minor logic” errors. For instance, the Validator will generate a Level 4 flag if the EMS Unit Arrival on Scene Time is earlier than the EMS Dispatch Time.

Level 1 and Level 2 flags must be resolved before the data file can be submitted to the NTDB. Level 3 and Level 4 flags indicate potential problems with a data file. The NTDB will still accept a file with Level 3 or 4 flags, but checking these fields and resolving any errors will help ensure the quality of your data.

For example, your NTDB Validator report flags a Level 3 error for ED/Hospital Arrival Date because it is greater than 30 days from the Injury Incident Date. Upon investigation, you might find that this field includes a typo that should be corrected. Alternatively, you might find that the date is correct as entered, because the acute trauma patient stayed at an outside hospital for 31 days and then transferred to your facility once stable enough to be closer to home. The point is to check the flag and correct the data if needed.

The NTDB Data Dictionary provides helpful guidance on resolving errors. Appendix 2 explains the meaning of each error flag for each data field. For example, say you run a Validator report on a patient file and it returns a Level 1 flag for the Cryoprecipitate Measurement field. Find that field in Appendix 2. There, you will see that a Level 1 flag for this field means that “the value is not a valid menu option.”

Q. How do you use “data cleaning” reports in trauma data validation?

While Validator reports are crucial to checking data submitted to the NTDB, they do not cover any non-NTDB data that your hospital is collecting. To validate your hospital-specific registry data, you need to run additional data cleaning reports.

Use your registry software’s reporting functionality to run reports on common problem areas. We recommend running two standard reports: blank fields and “Not Known/Not Recorded” (NK/NR) fields.

After you run these reports, step one is to resolve as many blanks and NK/NR fields as possible. For example, pre-hospital data fields are often blank because the EMS trip sheet is not available when the registrar abstracts the patient chart. The data cleaning report can be an effective reminder to go back and capture this information.

Step two is to use blank and NK/NR reports to identify performance improvement opportunities. Which fields are consistently blank or inappropriately marked as unknown? This can point to either a registrar problem or a documentation problem. It also indicates where further training or education is required.

Data cleaning reports can also be used to identify blanks that affect trauma type (blunt versus penetrating). Without this essential documentation, probability of survival — and thus your M and Z statistics — cannot be calculated. Registry managers can also use data cleaning reports to validate data on highest level of trauma activation and attending presence, which are critical for ACS compliance.

Q. How often should we run these registry reports?

Trauma registry managers or staff members should run data validation reports — and resolve identified errors — on a monthly basis.

Why? In our experience, many trauma programs check their registry data only once a year, when they submit data to the NTDB. Not uncommonly, the report flags more than 1,000 errors. Resolving all these errors can take days, creating last-minute problems when there is little time to spare.

Establishing a monthly reporting process enables the registry team to perform data validation in manageable batches. It gives staff time to focus on performance improvement opportunities, and it signifies a “closed” month with clean data.

Trauma Registry Q&A is a bimonthly column produced in partnership with Pomphrey Consulting, a comprehensive trauma registry company which provides training and trauma registry management. The American College of Surgeons has recognized our Trauma Registrar Mentorship Program for meeting the trauma registrar course requirement of CD15-7. Our course is revised each year to meet any and all ACS NTDB changes, as well as the demands of the ever-changing world of trauma registries. Our program also includes full trauma registrar training in ICD-10-CM and ICD-10-PCS. To visit our website, click here.