Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jul 13;5(4):60.
doi: 10.3390/mps5040060.

Performance and Information Leakage in Splitfed Learning and Multi-Head Split Learning in Healthcare Data and Beyond

Affiliations

Performance and Information Leakage in Splitfed Learning and Multi-Head Split Learning in Healthcare Data and Beyond

Praveen Joshi et al. Methods Protoc. .

Abstract

Machine learning (ML) in healthcare data analytics is attracting much attention because of the unprecedented power of ML to extract knowledge that improves the decision-making process. At the same time, laws and ethics codes drafted by countries to govern healthcare data are becoming stringent. Although healthcare practitioners are struggling with an enforced governance framework, we see the emergence of distributed learning-based frameworks disrupting traditional-ML-model development. Splitfed learning (SFL) is one of the recent developments in distributed machine learning that empowers healthcare practitioners to preserve the privacy of input data and enables them to train ML models. However, SFL has some extra communication and computation overheads at the client side due to the requirement of client-side model synchronization. For a resource-constrained client side (hospitals with limited computational powers), removing such conditions is required to gain efficiency in the learning. In this regard, this paper studies SFL without client-side model synchronization. The resulting architecture is known as multi-head split learning (MHSL). At the same time, it is important to investigate information leakage, which indicates how much information is gained by the server related to the raw data directly out of the smashed data-the output of the client-side model portion-passed to it by the client. Our empirical studies examine the Resnet-18 and Conv1-D architecture model on the ECG and HAM-10000 datasets under IID data distribution. The results find that SFL provides 1.81% and 2.36% better accuracy than MHSL on the ECG and HAM-10000 datasets, respectively (for cut-layer value set to 1). Analysis of experimentation with various client-side model portions demonstrates that it has an impact on the overall performance. With an increase in layers in the client-side model portion, SFL performance improves while MHSL performance degrades. Experiment results also demonstrate that information leakage provided by mutual information score values in SFL is more than MHSL for ECG and HAM-10000 datasets by 2×10-5 and 4×10-3, respectively.

Keywords: distributed collaborative machine learning; information leakage in distributed learning; multi-head split learning; parameter transmission-based distributed machine learning; privacy-preserving machine learning; split learning.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Multi-head-split-learning architecture.
Figure 2
Figure 2
Computation time (in seconds) for SFL and MHSL.
Figure 3
Figure 3
Mutual information score across the epochs for SFL and MHSL for the ECG dataset.
Figure 4
Figure 4
Mutual information score across the epochs for SFL and MHSL for three-channel datasets (a) HAM-10000 and (b) CIFAR-10.
Figure 5
Figure 5
Mutual information score across the epochs for SFL and MHSL for one-channel datasets (a) MNIST and (b) KMNIST.

Similar articles

Cited by

References

    1. Awotunde J.B., Adeniyi A.E., Ogundokun R.O., Ajamu G.J., Adebayo P.O. MIoT-based big data analytics architecture, opportunities and challenges for enhanced telemedicine systems. Enhanc. Telemed. e-Health. 2021;410:199–220.
    1. Chassang G. The impact of the EU general data protection regulation on scientific research. Ecancermedicalscience. 2017;11:709. doi: 10.3332/ecancer.2017.709. - DOI - PMC - PubMed
    1. Azzi A. The challenges faced by the extraterritorial scope of the General Data Protection Regulation. J. Intell. Prop. Info. Tech. Elec. Com. L. 2018;9:126.
    1. Qi A., Shao G., Zheng W. Assessing China’s cybersecurity law. Comput. Law Secur. Rev. 2018;34:1342–1354. doi: 10.1016/j.clsr.2018.08.007. - DOI
    1. Konečnỳ J., McMahan H.B., Yu F.X., Richtárik P., Suresh A.T., Bacon D. Federated learning: Strategies for improving communication efficiency. arXiv. 20161610.05492

LinkOut - more resources