Implementation of a Web-Based Organ Donation Educational Intervention: Development and Use of a Refined Process Evaluation Model

Abstract

Background: The lack of available organs is often considered to be the single greatest problem in transplantation today. Internet use is at an all-time high, creating an opportunity to increase public commitment to organ donation through the broad reach of Web-based behavioral interventions. Implementing Internet interventions, however, presents challenges including preventing fraudulent respondents and ensuring intervention uptake. Although Web-based organ donation interventions have increased in recent years, process evaluation models appropriate for Web-based interventions are lacking.

Objective: The aim of this study was to describe a refined process evaluation model adapted for Web-based settings and used to assess the implementation of a Web-based intervention aimed to increase organ donation among African Americans.

Methods: We used a randomized pretest-posttest control design to assess the effectiveness of the intervention website that addressed barriers to organ donation through corresponding videos. Eligible participants were African American adult residents of Georgia who were not registered on the state donor registry. Drawing from previously developed process evaluation constructs, we adapted reach (the extent to which individuals were found eligible, and participated in the study), recruitment (online recruitment mechanism), dose received (intervention uptake), and context (how the Web-based setting influenced study implementation) for Internet settings and used the adapted model to assess the implementation of our Web-based intervention.

Results: With regard to reach, 1415 individuals completed the eligibility screener; 948 (67.00%) were determined eligible, of whom 918 (96.8%) completed the study. After eliminating duplicate entries (n=17), those who did not initiate the posttest (n=21) and those with an invalid ZIP code (n=108), 772 valid entries remained. Per the Internet protocol (IP) address analysis, only 23 of the 772 valid entries (3.0%) were within Georgia, and only 17 of those were considered unique entries and could be considered for analyses. With respect to recruitment, 517 of the 772 valid entries (67.0%) of participants were recruited from a Web recruiter. Regarding dose received, no videos from the intervention website were watched in their entirety, and the average viewing duration was 17 seconds over the minimum. With respect to context, context analysis provided us with valuable insights into factors in the Internet environment that may have affected study implementation. Although only active for a brief period of time, the Craigslist website advertisement may have contributed the largest volume of fraudulent responses.

Conclusions: We determined fraud and low uptake to be serious threats to this study and further confirmed the importance of conducting a process evaluation to identify such threats. We suggest checking participants' IP addresses before study initiation, selecting software that allows for automatic duplicate protection, and tightening minimum requirements for intervention uptake. Further research is needed to understand how process evaluation models can be used to monitor implementation of Web-based studies.

Keywords: Internet; evaluation methodology; health education; intervention; organ donation; program evaluation; research techniques.

Figure 1

Intervention (Project WEB ACTS) interactive website, Fairness In Healthcare Delivery module.

Figure 2

Summary of eliminated data through Reach assessment.