An Evaluation of Local Implementing Partner Performance During the First 2 Years of the USAID/PEPFAR Transition

Abstract

Introduction: Locally led and owned development is considered the best practice for international aid. As an implementing agency for the U.S. President's Emergency Plan for AIDS Relief (PEPFAR), the U.S. Agency for International Development (USAID) supported the goal of transitioning 70% of its portfolio funding directly to local organizations by 2020, including partner country governments. However, limited evidence or evaluation exists on how such a transition can help achieve HIV-related health outcomes.

Methods: We evaluated monitoring, evaluation, and reporting performance; calculated indicators; and quality of service across the HIV/AIDS treatment cascade for local and international partners in the USAID/PEPFAR portfolio implementing similar programs during the U.S. Government fiscal years (FY) 2019 to 2020 (October 1, 2018-September 30, 2020). We compared results aggregated globally, by country, and across individual partners.

Results: Globally, local partners met a lower proportion of their treatment targets than international partners and did not meet targets for pre-exposure prophylaxis or voluntary medical male circumcision in FY2020. However, local partners exceeded targets in programs supporting orphans, vulnerable children, and key populations affected by HIV/AIDS. Local partners also had testing positivity, linkage rates, and viral load suppression that were equivalent to or higher than that of international partners. Based on available assessments, local partners displayed quality of service delivery comparable to international partners.

Conclusion: Local partners faced challenges, including unfamiliarity with USAID funding, increasing targets across several indicators, and the syndemics of HIV/AIDS and COVID-19. A higher percentage of targets and funding led South African local partners to yield an outsized effect on global percent target achievement. While these findings should be interpreted cautiously due to limited sample size and short time horizon, they are a key first step in evaluating the local partner transition support of the long-term goal of sustained epidemic control of HIV/AIDS.

FIGURE 1

USAID Targets for Number of Clients Currently Receiving Treatment by Operating Unit^a Abbreviations: FY, fiscal year; USAID, U.S. Agency for International Development. ^aUSAID target volume and percent target share are displayed for each operating unit for each FY. Cell size and shading is in relation to the proportion of each operating unit currently receiving treatment targets.

FIGURE 2

Globally Aggregated Achievement of Programmatic Targets in Key Technical Areas of (A) Treatment and (B) Prevention and Support for USAID/PEPFAR International vs. Local Partners Across 23 LTS Country Programs^a Abbreviations: FY, fiscal year; OVC, orphans and vulnerable children; PEPFAR, U.S. President’s Emergency Plan for AIDS Relief; LTS, long-term strategy; PrEP, pre-exposure prophylaxis; USAID, U.S. Agency for International Development; VMMC, voluntary medical male circumcision. ^aThe horizontal line corresponding to each bar indicates the target set before implementation. The percentage over each bar indicates the percent target achievement during that implementation year. The y-axis for each indicator is scaled to the total number of targets to be achieved.

FIGURE 3

Achievement of Programmatic Targets in Key Technical Areas of (A) Treatment and (B) Prevention and Support for USAID/PEPFAR International vs. Local Partners Across 22 LTS Country Programs Excluding South Africa^a Abbreviations: FY, fiscal year; PEPFAR, U.S. President’s Emergency Plan for AIDS Relief; LTS, long-term strategy; USAID, U.S. Agency for International Development. ^aThe horizontal line corresponding to each bar indicates the target set before implementation. The percentage over each bar indicates the percent target achievement during that implementation year. The y-axis for each indicator is scaled to the total number of targets to be achieved.

FIGURE 4

Achievement of Programmatic Targets in Key Technical Areas of (A) Treatment and (B) Prevention and Support for USAID/PEPFAR International vs. Local Partners in South Africa Abbreviations: FY, fiscal year; OVC, orphans and vulnerable children; PEPFAR, U.S. President’s Emergency Plan for AIDS Relief; LTS, long-term strategy; PrEP, pre-exposure prophylaxis; USAID, U.S. Agency for International Development; VMMC, voluntary medical male circumcision. ^aThe horizontal line corresponding to each bar indicates the target set before implementation. The percentage over each bar indicates the percent target achievement during that implementation year. The y-axis for each indicator is scaled to the total number of targets to be achieved.

FIGURE 5

Achievement of Targets Aggregated at the (A) Operating Unit Level or (B) Individual Partner Level for International and Local Partners^a Abbreviations: KP, key populations; OU, operating unit; OVC, orphans and vulnerable children; PrEP, pre-exposure prophylaxis; VMMC, voluntary medical male circumcision. ^aPercent achievement is displayed on a log scale. The first and third quartiles (the 25th and 75th percentiles) are displayed as the lower and upper hinges. Each whisker extends from the hinge to the highest or lowest value no further than 1.5 times the interquartile range from the respective hinge. Individual OU or partner percent achievements are shown as lighter, jittered points with the size of the jittered circle corresponding to the raw results achieved by each OU or implementing partner, respectively.

FIGURE 6

Calculated Indicators of Individual Partners Disaggregated by FY and Partner Type: (A) Test Positivity, (B) Linkage to Treatment, (C) Viral Load Coverage, and (D) Viral Load Suppression^a Abbreviation: FY, fiscal year. ^aThe first and third quartiles (the 25th and 75th percentiles) are displayed as the lower and upper hinges. Each whisker extends from the hinge to the highest or lowest value no further than 1.5 times the interquartile range from the respective hinge. Individual partner linkages are shown as lighter, jittered points. The dashed line corresponds to target thresholds for these indicators: 95% for linkage and viral load suppression, 80% for viral load coverage. Y-axes are scaled differently for each indicator. Only partners reporting both number of clients testing positive and number of clients currently receiving treatment were included for linkage calculations. Regional partners were classified as local across both fiscal years.

FIGURE 7

Service Quality of USAID/PEPFAR International vs. Local Partners (A) Across All Sites and (B) by Select SIMS Categories^a Abbreviations: AGYW, adolescent girls and young women; CEE, core essential element; FY, fiscal year; GBV, gender-based violence; MER, monitoring, evaluation, and reporting; OVC, orphans and vulnerable children; PEPFAR, U.S. President’s Emergency Plan for AIDS Relief; SIMS, Site Improvement Through Monitoring System. ^aThe scores on SIMS are coded as “needs urgent remediation” (Red), “needs improvement” (Yellow), and “meets standard” (Green). Numbers under the percentages in each category respond to CEE scores, not to the number of site assessments; multiple CEE scores are assessed per each site visit. Of note, SIMS CEEs measure quality metrics at a facility, community, or above site level, while MER indicators are measures of outputs at a partner level, thus MER indicators do not align exactly with SIMS categories but may inform multiple CEEs. The specific CEEs assessed for each SIMS category are available in the . Current as of June 2021.