. 2018 Oct 24;18(1):814.

doi: 10.1186/s12913-018-3626-4.

Evaluating diverse electronic consultation programs with a common framework

Delphine S Tuot^{1

2}, Clare Liddy^{3

4}, Varsha G Vimalananda^{5

6}, Jennifer Pecina⁷, Elizabeth J Murphy^{8

9}, Erin Keely^{10

11}, Steven R Simon^{12

13}, Frederick North¹⁴, Jay D Orlander^{6

13}, Alice Hm Chen^{8

9}

Affiliations

¹ Center for Innovation in Access and Quality at Priscilla Chan and Mark Zuckerberg San Francisco General Hospital and Trauma Center, University of California San Francisco, San Francisco, CA, 94110, USA. Delphine.tuot@ucsf.edu.
² Deparment of Medicine, University of California, San Francisco, San Francisco, CA, USA. Delphine.tuot@ucsf.edu.
³ C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, ON, Canada.
⁴ Department of Family Medicine, University of Ottawa, Ottawa, ON, Canada.
⁵ Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial VA Medical Center, Bedford, MA, USA.
⁶ Department of Medicine, Boston University School of Medicine, Boston, MA, USA.
⁷ Department of Family Medicine, Mayo Clinic, Rochester, MN, USA.
⁸ Center for Innovation in Access and Quality at Priscilla Chan and Mark Zuckerberg San Francisco General Hospital and Trauma Center, University of California San Francisco, San Francisco, CA, 94110, USA.
⁹ Deparment of Medicine, University of California, San Francisco, San Francisco, CA, USA.
¹⁰ Department of Medicine, University of Ottawa, Ottawa, ON, Canada.
¹¹ The Ottawa Hospital, Ottawa, ON, Canada.
¹² Harvard Medical School, Boston, USA.
¹³ VA Boston Healthcare System, Boston, USA.
¹⁴ Department of Primary Care Internal Medicine, Mayo Clinic, Rochester, MN, USA.

PMID: 30355346
PMCID: PMC6201558
DOI: 10.1186/s12913-018-3626-4

Evaluating diverse electronic consultation programs with a common framework

Delphine S Tuot et al. BMC Health Serv Res. 2018.

. 2018 Oct 24;18(1):814.

doi: 10.1186/s12913-018-3626-4.

Authors

Affiliations

¹ Center for Innovation in Access and Quality at Priscilla Chan and Mark Zuckerberg San Francisco General Hospital and Trauma Center, University of California San Francisco, San Francisco, CA, 94110, USA. Delphine.tuot@ucsf.edu.
² Deparment of Medicine, University of California, San Francisco, San Francisco, CA, USA. Delphine.tuot@ucsf.edu.
³ C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, ON, Canada.
⁴ Department of Family Medicine, University of Ottawa, Ottawa, ON, Canada.
⁵ Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial VA Medical Center, Bedford, MA, USA.
⁶ Department of Medicine, Boston University School of Medicine, Boston, MA, USA.
⁷ Department of Family Medicine, Mayo Clinic, Rochester, MN, USA.
⁸ Center for Innovation in Access and Quality at Priscilla Chan and Mark Zuckerberg San Francisco General Hospital and Trauma Center, University of California San Francisco, San Francisco, CA, 94110, USA.
⁹ Deparment of Medicine, University of California, San Francisco, San Francisco, CA, USA.
¹⁰ Department of Medicine, University of Ottawa, Ottawa, ON, Canada.
¹¹ The Ottawa Hospital, Ottawa, ON, Canada.
¹² Harvard Medical School, Boston, USA.
¹³ VA Boston Healthcare System, Boston, USA.
¹⁴ Department of Primary Care Internal Medicine, Mayo Clinic, Rochester, MN, USA.

PMID: 30355346
PMCID: PMC6201558
DOI: 10.1186/s12913-018-3626-4

Abstract

Background: Electronic consultation is an emerging mode of specialty care delivery that allows primary care providers and their patients to obtain specialist expertise without an in-person visit. While studies of individual programs have demonstrated benefits related to timely access to specialty care, electronic consultation programs have not achieved widespread use in the United States. The lack of common evaluation metrics across health systems and concerns related to the generalizability of existing evaluation efforts may be hampering further growth. We sought to identify gaps in knowledge related to the implementation of electronic consultation programs and develop a set of shared evaluation measures to promote further diffusion.

Methods: Using a case study approach, we apply the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) and the Quadruple Aim frameworks of evaluation to examine electronic consultation implementation across diverse delivery systems. Data are from 4 early adopter healthcare delivery systems (San Francisco Health Network, Mayo Clinic, Veterans Administration, Champlain Local Health Integration Network) that represent varied organizational structures, care for different patient populations, and have well-established multi-specialty electronic consultation programs. Data sources include published and unpublished quantitative data from each electronic consultation database and qualitative data from systems' end-users.

Results: Organizational drivers of electronic consultation implementation were similar across the systems (challenges with timely and/or efficient access to specialty care), though unique system-level facilitators and barriers influenced reach, adoption and design. Effectiveness of implementation was consistent, with improved patient access to timely, perceived high-quality specialty expertise with few negative consequences, garnering high satisfaction among end-users. Data about patient-specific clinical outcomes are lacking, as are policies that provide guidance on the legal implications of electronic consultation and ideal remuneration strategies.

Conclusion: A core set of effectiveness and implementation metrics rooted in the Quadruple Aim may promote data-driven improvements and further diffusion of successful electronic consultation programs.

Keywords: E-consult; Electronic consultation; Evaluation; Quadruple aim; RE-AIM.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

This study was approved by the University of California, San Francisco Institutional Review Board (study number: 11–05827). This study did not include direction patient contact and thus formal consent was not obtained.

Consent for publication

N/A.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Cited by

Barriers and facilitators for implementation of electronic consultations (eConsult) to enhance access to specialist care: a scoping review.
Osman MA, Schick-Makaroff K, Thompson S, Bialy L, Featherstone R, Kurzawa J, Zaidi D, Okpechi I, Habib S, Shojai S, Jindal K, Braam B, Keely E, Liddy C, Manns B, Tonelli M, Hemmelgarn B, Klarenbach S, Bello AK. Osman MA, et al. BMJ Glob Health. 2019 Sep 13;4(5):e001629. doi: 10.1136/bmjgh-2019-001629. eCollection 2019. BMJ Glob Health. 2019. PMID: 31565409 Free PMC article.
General Practitioners' Perceptions of Whether Teleconsultations Reduce the Number of Face-to-face Visits in the Catalan Public Primary Care System: Retrospective Cross-Sectional Study.
López Seguí F, Vidal-Alaball J, Sagarra Castro M, García-Altés A, García Cuyàs F. López Seguí F, et al. J Med Internet Res. 2020 Mar 16;22(3):e14478. doi: 10.2196/14478. J Med Internet Res. 2020. PMID: 32175914 Free PMC article.
Implementing an Opt-in eConsult Program at Seven Academic Medical Centers: a Qualitative Analysis of Primary Care Provider Experiences.
Deeds SA, Dowdell KJ, Chew LD, Ackerman SL. Deeds SA, et al. J Gen Intern Med. 2019 Aug;34(8):1427-1433. doi: 10.1007/s11606-019-05067-7. Epub 2019 Jun 13. J Gen Intern Med. 2019. PMID: 31197734 Free PMC article.
Implementation fidelity of the Systems for Person-Centered Elder Care (SPEC): a process evaluation study.
Choi H, Jung YI, Kim H. Choi H, et al. Implement Sci. 2021 May 12;16(1):52. doi: 10.1186/s13012-021-01113-3. Implement Sci. 2021. PMID: 33980251 Free PMC article.
Implementing a general practitioner-to-general physician eConsult service (eConsultant) in Australia.
Job J, Nicholson C, Calleja Z, Jackson C, Donald M. Job J, et al. BMC Health Serv Res. 2022 Oct 24;22(1):1278. doi: 10.1186/s12913-022-08663-2. BMC Health Serv Res. 2022. PMID: 36280832 Free PMC article.

See all "Cited by" articles

References

1. Dall Tim WT, Ritashree C, Will I. The Complexities of Physician Supply and Demand: Projections from 2014 to 2025. Washginton, DC: IHS Inc. for the Association of American Medical Colleges; 2016.
1. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573–576. doi: 10.1370/afm.1713. - DOI - PMC - PubMed
1. Olayiwola JN, Anderson D, Jepeal N, Aseltine R, Pickett C, Yan J, et al. Electronic consultations to improve the primary care-specialty care Interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med. 2016;14(2):133–140. doi: 10.1370/afm.1869. - DOI - PMC - PubMed
1. Cruz ML, Gleason N, Wang M, Wrenn K, Gonzales R. Transforming the endocrine consult: asynchronous provider consultations. Endocr Pract. 2015;21(5):514–521. doi: 10.4158/EP14297.OR. - DOI - PubMed
1. Tuot DS, Murphy EJ, McCulloch CE, Leeds K, Chan E, Chen AH. Leveraging an electronic referral system to build a medical neighborhood. Healthc (Amst). 2015;3(4):202–208. doi: 10.1016/j.hjdsi.2015.04.001. - DOI - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

P-1606-08494/BlueCross BlueShield of South Carolina Foundation

LinkOut - more resources

Full Text Sources
Medical
- MedlinePlus Health Information
- The YODA Project

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Evaluating diverse electronic consultation programs with a common framework

Affiliations

Evaluating diverse electronic consultation programs with a common framework

Authors

Affiliations

Abstract

Conflict of interest statement

Ethics approval and consent to participate

Consent for publication

Competing interests

Publisher’s Note

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Medical