Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Sep/Oct;11(5):440-445.
doi: 10.1177/1941738119854759. Epub 2019 Jul 2.

Lessons on Data Collection and Curation From the NFL Injury Surveillance Program

Nancy A Dreyer  1 , Christina D Mack  2 , Robert B Anderson  3   4 , Edward M Wojtys  5 , Elliott B Hershman  6 , Allen Sills  7
Affiliations

Lessons on Data Collection and Curation From the NFL Injury Surveillance Program

Nancy A Dreyer et al. Sports Health. 2019 Sep/Oct.

Abstract

Background: "Research-ready" evidence platforms that link sports data with anonymized electronic health records (EHRs) or other data are important tools for evaluating injury occurrence in response to changes in games, training, rules, and other factors. While there is agreement that high-quality data are essential, there is little evidence to guide data curation.

Hypothesis: We hypothesized that an EHR used in the course of clinical care and curated for research readiness can provide a robust evidence platform. Our purpose was to describe the data curation used for active injury surveillance by the National Football League (NFL).

Study design: Dynamic cohort study.

Level of evidence: Level 2.

Methods: Players provide informed consent for research activities through the collective bargaining process. A league-wide EHR is used to record injuries that come to the attention of the teams' athletic trainers and physicians, NFL medical spotters, or unaffiliated neurotrauma consultants. Information about football activities and injuries are linkable by player, setting, and event to other sports-related data, including game statistics and game-day stadium quality measures, using a unique player identification designed to protect player privacy. Ongoing data curation is used to review data completeness and accuracy and is adjusted over time in response to findings.

Results: The core data curation activities include monthly injury summaries to team staff, queries to resolve incomplete reporting, and periodic external checks. Experiences derived from producing more than 100 reports per year on diverse topics are used to update coding training and related guidance documents in response to missing data or inconsistent coding that is observed. Roughly 20% more injuries were recorded for the same "reportable" injuries after switching from targeted reporting to an EHR.

Conclusion: Research-ready databases need systematic curation for quality and completeness, along with related action plans. More injuries were reported through EHR than through targeted reporting.

Clinical relevance: Evidence-driven decision-making thrives on reliable data fine-tuned through systematic use, review, and ongoing adjustments to the curation process.

Keywords: data curation; electronic health records; epidemiology surveillance; football.

PubMed Disclaimer

Conflict of interest statement

The following authors declared potential conflicts of interest: N.A.D. and C.D.M. are employed by IQVIA; R.B.A. received consulting fees from Amniox, Artelon, Arthrex, Bioventus, DJO, Wright Medical, and Zimmer Biomet, and royalties from Arthrex, Biomet, Wright Medical, and Zimmer Biomet; E.M.W. received travel payments from the National Football League (NFL) and AOSSM and payments from the NFL and NIAMS/KAI, he also serves as editor in chief for Sports Health; E.B.H. received consulting fees and royalties from Active Implants Corp and royalties from Zimmer Biomet; and A.S. is employed by the NFL. This research program was funded by the NFL and approved by the National Football League Players Association under a collectively bargained agreement pertaining to NFL player research. The authors assert that these results are presented clearly, honestly, and without fabrication, falsification, or inappropriate data manipulation.

Figures

Figure 1.
Figure 1.
Evolution of data collection for injury surveillance in the National Football League (NFL). MRSA, methicillin-resistant Staphylococcus aureus.
Figure 2.
Figure 2.
National Football League (NFL) evidence platform for player health and safety. ATC, certified athletic trainer; UNC, unaffiliated neurotrauma consultant.
See this image and copyright information in PMC

References

    1. Califf RM. Benefit-risk assessments at the US Food and Drug Administration: finding the balance. JAMA. 2017;317:693-694. - PubMed
    1. Califf RM, Robb MA, Bindman AB, et al. Transforming evidence generation to support health and health care decisions. N Engl J Med. 2016;375:2395-2400. - PubMed
    1. Center for Devices and Radiological Health, Center for Biologics Evaluation and Research, U.S. Food and Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices. Guidance for industry and Food and Drug Administration staff. US Food and Drug Administration; August, 2017. https://www.fda.gov/regulatory-information/search-fda-guidance-documents.... Accessed June 16, 2019.
    1. Daniel G, Silcox C, Bryan J, et al. White paper: Characterizing RWD Quality and Relevancy for Regulatory Purposes. Published October 1, 2018. https://healthpolicy.duke.edu/sites/default/files/atoms/files/characteri.... Accessed June 16, 2019.
    1. Dreyer NA. Advancing a framework for regulatory use of real-world evidence: when real is reliable. Ther Innov Regul Sci. 2018;52:362-368. - PMC - PubMed