Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Jun 25;1(3):276-287.
doi: 10.1016/j.mcpdig.2023.05.010. eCollection 2023 Sep.

Continuous Improvement of Digital Health Applications Linked to Real-World Performance Monitoring: Safe Moving Targets?

Stephen Gilbert  1   2 , Andreia Pimenta  2 , Ashley Stratton-Powell  3 , Cindy Welzel  1 , Tom Melvin  4
Affiliations
Review

Continuous Improvement of Digital Health Applications Linked to Real-World Performance Monitoring: Safe Moving Targets?

Stephen Gilbert et al. Mayo Clin Proc Digit Health. .

Abstract

Real-time high-quality data on the performance of digital health applications is needed for feedback-led optimization and to ensure safety and performance, particularly if they will have on-market updates. Developers must verify that applications accurately and consistently fulfill their intended purpose in real-world use. In particular, new thinking from regulators recognizes the importance of monitoring real-world performance. It is acknowledged that real-world data can deliver information from wider patient populations than are generally included in controlled studies, and in certain circumstances, this can enable extensions of the application's intended purpose. Proactive postmarket surveillance surveys are an important source of real-world data that are distinct from clinical investigations but may vary in quality and, if inappropriately designed, can be subject to uncontrolled bias. We aimed to describe the practice of real-world data gathering through patient-reported and clinician-reported outcomes and high-quality surveys and identify challenges, uncertainties, and health policy gaps.

PubMed Disclaimer

Conflict of interest statement

Dr Gilbert has or has had consulting relationships with Una Health GmbH, Lindus Health Ltd, FLO Ltd, Thymia Ltd, and Ada Health GmbH and holds share options in Ada Health GmbH. Dr Pimenta is an employee of Ada Health GmbH. Dr Stratton-Powell is an employee of RQM+. Dr Welzel declares no competing interests. Dr Melvin is an unpaid advisory board member of Pumpinheart Ltd; previously, a senior medical officer in medical devices at the Health Products Regulatory Authority, Ireland; and a previous co-chair of the Clinical Investigation and Evaluation Working Group of the European Commission.

Figures

Figure 1
Figure 1
Overview of software in health care and risk categories of Software as Medical Device (SaMD) under the international IMDRF framework, which is reflected in the regulations of most countries. IMDRF, International Medical Device Regulators Forum.
Figure 2
Figure 2
The concept of continuous monitoring, continuous improvement, continuous approval, and continuous release of DHAs, which has been introduced in recent thinking from regulatory bodies.,
Figure 3
Figure 3
Aspects to consider in survey validation.
See this image and copyright information in PMC

References

    1. Marwaha J.S., Landman A.B., Brat G.A., Dunn T., Gordon W.J. Deploying digital health tools within large, complex health systems: key considerations for adoption and implementation. NPJ Digit Med. 2022;5(1):13. - PMC - PubMed
    1. Wilson K., Bell C., Wilson L., Witteman H. Agile research to complement agile development: a proposal for an mHealth research lifecycle. NPJ Digit Med. 2018;1:46. - PMC - PubMed
    1. IMDRF Software as a Medical Device (SaMD) Working Group . IMDRF; 2014. ‘Software as a Medical Device’: Possible Framework for Risk Categorization and Corresponding Considerations.
    1. Medical Device Coordination Group. Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746–IVDR. MDCG; 2019. md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf.
    1. Essén A., Stern A.D., Haase C.B., et al. Health app policy: international comparison of nine countries’ approaches. NPJ Digit Med. 2022;5(1):31. - PMC - PubMed

LinkOut - more resources