The MIMIC Code Repository: enabling reproducibility in critical care research

Abstract

Objective: Lack of reproducibility in medical studies is a barrier to the generation of a robust knowledge base to support clinical decision-making. In this paper we outline the Medical Information Mart for Intensive Care (MIMIC) Code Repository, a centralized code base for generating reproducible studies on an openly available critical care dataset.

Materials and methods: Code is provided to load the data into a relational structure, create extractions of the data, and reproduce entire analysis plans including research studies.

Results: Concepts extracted include severity of illness scores, comorbid status, administrative definitions of sepsis, physiologic criteria for sepsis, organ failure scores, treatment administration, and more. Executable documents are used for tutorials and reproduce published studies end-to-end, providing a template for future researchers to replicate. The repository's issue tracker enables community discussion about the data and concepts, allowing users to collaboratively improve the resource.

Discussion: The centralized repository provides a platform for users of the data to interact directly with the data generators, facilitating greater understanding of the data. It also provides a location for the community to collaborate on necessary concepts for research progress and share them with a larger audience. Consistent application of the same code for underlying concepts is a key step in ensuring that research studies on the MIMIC database are comparable and reproducible.

Conclusion: By providing open source code alongside the freely accessible MIMIC-III database, we enable end-to-end reproducible analysis of electronic health records.

Keywords: critical care; data mining; electronic health record; intensive care; mimic-iii; reproducibility.

Figure 1.

Comparison of severity of illness score distributions.

Figure 2.

Comparison of areas under the receiver operating curve for SOFA scores calculated from MIMIC code and a prior research report.

Figure 3.

Logic behind the query for converting aperiodically recorded ventilator settings into durations of mechanical ventilation.

Figure 4.

Example of a patient who was both mechanically ventilated and receiving vasopressors for cardiovascular support.

Figure 5.

Comparison of 3 methods for calculating presence of a comorbidity for a patient using billing data: an updated coding from the AHRQ which uses diagnosis-related group (DRG) codes to mask non-comorbid conditions, the same coding without the DRG masking, and finally an alternative coding that does not use DRG masking, proposed by Quan et al.

Figure 6.

Example of a notebook providing a tutorial with MIMIC-III data.