Distributed Statistical Analyses: A Scoping Review and Examples of Operational Frameworks Adapted to Health Analytics

Abstract

Background: Data from multiple organizations are crucial for advancing learning health systems. However, ethical, legal, and social concerns may restrict the use of standard statistical methods that rely on pooling data. Although distributed algorithms offer alternatives, they may not always be suitable for health frameworks.

Objective: This study aims to support researchers and data custodians in three ways: (1) providing a concise overview of the literature on statistical inference methods for horizontally partitioned data, (2) describing the methods applicable to generalized linear models (GLMs) and assessing their underlying distributional assumptions, and (3) adapting existing methods to make them fully usable in health settings.

Methods: A scoping review methodology was used for the literature mapping, from which methods presenting a methodological framework for GLM analyses with horizontally partitioned data were identified and assessed from the perspective of applicability in health settings. Statistical theory was used to adapt methods and derive the properties of the resulting estimators.

Results: From the review, 41 articles were selected and 6 approaches were extracted to conduct standard GLM-based statistical analysis. However, these approaches assumed evenly and identically distributed data across nodes. Consequently, statistical procedures were derived to accommodate uneven node sample sizes and heterogeneous data distributions across nodes. Workflows and detailed algorithms were developed to highlight information sharing requirements and operational complexity.

Conclusions: This study contributes to the field of health analytics by providing an overview of the methods that can be used with horizontally partitioned data by adapting these methods to the context of heterogeneous health data and clarifying the workflows and quantities exchanged by the methods discussed. Further analysis of the confidentiality preserved by these methods is needed to fully understand the risk associated with the sharing of summary statistics.

Keywords: GLMs; algorithms; data custodians; data science; distributed algorithms; distributed analysis; federated analysis; generalized linear models; horizontally partitioned data; learning health systems; review methods; scoping; searches; statistics; synthesis.

Figure 1.. Article selection process for the scoping review. Detailed inclusion and exclusion criteria are described in the text and in the protocol.

Figure 2.. Workflow I: each node calculates summary statistics from its own samples. Results are sent to the coordinating center, which combines the information provided by each node to produce the final estimates.

Figure 3.. Workflow II: multiple communication rounds are allowed between the coordinating center and the data storage nodes.

Figure 4.. Workflow III: multiple communication rounds are allowed between the coordinating center and the data storage nodes, with node 1 following a distinct communication pattern compared to the other nodes.

Figure 5.. Workflow IV: multiple communication rounds are allowed between the coordinating center (CC) and the data storage nodes, with 2 back-and-forth distinct communication exchanges between each node and the CC at each iteration.