The MSOAC approach to developing performance outcomes to measure and monitor multiple sclerosis disability

Abstract

Background: The Multiple Sclerosis Outcome Assessments Consortium (MSOAC) was formed by the National MS Society to develop improved measures of multiple sclerosis (MS)-related disability.

Objectives: (1) To assess the current literature and available data on functional performance outcome measures (PerfOs) and (2) to determine suitability of using PerfOs to quantify MS disability in MS clinical trials.

Methods: (1) Identify disability dimensions common in MS; (2) conduct a comprehensive literature review of measures for those dimensions; (3) develop an MS Clinical Data Interchange Standards Consortium (CDISC) data standard; (4) create a database of standardized, pooled clinical trial data; (5) analyze the pooled data to assess psychometric properties of candidate measures; and (6) work with regulatory agencies to use the measures as primary or secondary outcomes in MS clinical trials.

Conclusion: Considerable data exist supporting measures of the functional domains ambulation, manual dexterity, vision, and cognition. A CDISC standard for MS ( http://www.cdisc.org/therapeutic#MS ) was published, allowing pooling of clinical trial data. MSOAC member organizations contributed clinical data from 16 trials, including 14,370 subjects. Data from placebo-arm subjects are available to qualified researchers. This integrated, standardized dataset is being analyzed to support qualification of disability endpoints by regulatory agencies.

Keywords: MS disability; clinical trial database; data standards; performance outcome measures; regulatory qualification.

Figure 1.

Overview of literature review results. The literature review was performed in three consecutive levels. For Level 2, the workgroup applied a structured algorithm to review abstracts and categorize each article into “Included” or “Excluded” (see Abstract Filtering Criteria in Supplementary Table 3), with reasons for exclusion. Abstracts that included a measure of cognition received a code of “keep” or “exclude” only. Abstracts that included measures of vision, manual dexterity, or ambulation received a code of “high,” “medium,” and “low” importance or “exclude.” All articles meeting the inclusion criteria were entered into the Data Extraction Table (Supplementary Table 4) and ranked to identify articles of most relevance to study objectives. Additional exclusions were also identified at this stage. For those articles that met the inclusion criteria, subject matter experts (SMEs) carried out the initial analysis and a separate group reviewed the analysis. A meeting was held to adjudicate any differences in the determinations of the SMEs. The Data Extraction Table (Supplementary Table 4) contains information on a total of 564 reviewed publications.

Figure 2.

A concept map representing relapse in MS. Multiple pathways can lead to the conclusion that relapse has occurred in patients with MS, including variations in relapse criteria and criteria for determination of severity. These criteria are typically anchored on changes in EDSS score. The resulting data standard accommodates this variation and specifies where in the CDISC data model (SDTM) this information can be found (represented by yellow boxes). CE is the Clinical Events domain; TM is the Trial Milestones domain; TS is the Trial Summary domain. Symbols correspond to the Bioinformatics Research Information Domain Grid (BRIDG) model concept classes (not discussed). Stars in orange represent the “Observation Result” class; circles in blue represent the “Assessor” class; and pentagons in yellow represent SDTM domains.

Figure 3.

Process used for development of CDISC data standards for MS, v1.0. A working group of MS subject matter experts and CDISC data standards experts reviewed the NINDS Common Data Elements for MS (https://www.commondataelements.ninds.nih.gov/MS.aspx#tab=Data_StandardsNLM/NIH). Concepts extracted from these CDEs were either retained or eliminated based on their relevance and whether or not they were already represented in CDISC SDTM standards. The concepts retained formed the scope for v1.0 of the MS CDISC User Guide, and those were further developed by CDISC data modelers in consultation with clinical SMEs. Development work included concept maps, data modeling examples, and controlled terminology. Controlled terminology was developed in collaboration with the NCI Enterprise Vocabulary Service (EVS). Examples of controlled terminology developed for MS include more than 500 terms registered in support of coding the various items and scores for the clinical outcome assessments and functional tests developed as a part of this project. The draft user guide was assembled as the output of this working group.

Figure 4.

Steps in data mapping. Data contributors agreed to provide data that meet the requirement of either a “Limited Dataset” or a “De-identified Dataset” in accordance with the Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor requirements (45 CFR 164 Subpart E, https://www.gpo.gov/fdsys/pkg/CFR-2011-title45-vol1/pdf/CFR-2011-title45-vol1-part164.pdf). C-Path policy is to exceed that minimum requirement whenever possible by further de-identifying the data after receiving it so that, in addition to meeting HIPAA Safe Harbor requirements, the data do not contain the year portion (which is allowed by Safe Harbor) of any dates. Instead, the timing of events is represented as time in relation to the study medication start date (defined as study day 1). Data were standardized using the CDISC Study Data Tabulation Model (SDTM). This process was carried out by C-Path data managers in three stages: (1) logically planning how the data would fit into SDTM, (2) programmatically remapping the data, and (3) validating the mapped data. The purpose of the validation step was two-fold: to ensure that the meaning of the data was accurately preserved and to check for SDTM compliance. The latter was done with the aid of the Pinnacle 21 Validator tool. Once all the data were in SDTM format, the data were pooled together into an aggregated dataset that was provided to the CRO, Premier Research.

Figure 5.

Baseline descriptive statistics for the pooled subjects in the MSOAC Database. Through the data contribution agreement, contributors specified the level of access to the data (e.g. C-Path staff and contractors, MSOAC members) and which data they will contribute. Some contributors provided all of the data collected for a given study while others provided only the data relevant to the MSOAC analyses. Consequently, some information, including demographics on race and geographic region, is missing.

Figure 6.

Baseline descriptive statistics for the pooled subjects in the placebo-arm database. The data contribution agreement for each study stated whether the placebo-arm data could be made available to qualified researchers. For the studies where permission was granted, the standardized records for the placebo-arm subjects were copied and pooled together into a separate dataset. These records were further anonymized by removing the study IDs and using a random number generator to assign all of the subjects a new ID number. To request access to the database, use the following link: https://c-path.org/programs/msoac/.