An ecosystem of community members and experts to track multidimensional equity in healthcare coverage and health outcomes

Abstract

The VERSE project created a global network of researchers and practitioners with a shared goal: to develop new ways to holistically measure equity in healthcare coverage and health outcomes and to use these measures to inform local healthcare policy.

Introduction

Led by Assistant Professor Bryan Patenaude, the VERSE team at Johns Hopkins University developed both a multivariate equity metric and assessment procedure focused on immunization coverage and an approach to identify potential contributors to inequity. The analysis, statistical package and methods to identify contributors to inequity form the VERSE Equity Toolkit, an open-access R-based kit tailored to run with any demographic and health survey (DHS) or individual-level dataset. VERSE partners co-developed the toolkit and use it to answer specific research questions and inform policy. Makerere University and the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b) contributed to the design and validation of the toolkit. Makerere University customized the toolkit via programming its applications to different sets of data, sourced from the government and its partners, and conducting equity assessments of the Ugandan health system. Icddr,b collected household data on immunization coverage and healthcare service access in urban slums to document equity in a sample typically under-represented in large national surveys. Peking University customized the toolkit using data from the Chinese CDC to assess equity in coverage of non-immunization program vaccines to prioritize vaccines for program coverage.

Context

Since 2015, a successful collaboration between Makerere University, icddr,b, and the team in Baltimore has developed a series of indicators on the economic burden of vaccine-preventable diseases and on preferences for various prevention measures – otherwise known as the Decade of Vaccine Economics (DOVE) project. Partners co-developed the protocol and led primary data collection. The results were featured in presentations from the partners to the Ugandan and Bangladeshi ministries of health and regional health offices, helping to advance immunization efforts and supplement analysts’ administrative data to monitor health system performance. Following the World Health Organization’s call for a methodology to monitor immunization coverage equity in line with the 2030 Agenda for Sustainable Development, the Baltimore team developed a standardized approach for measuring multivariate equity in vaccine coverage and health outcomes. In consultation with their partners, they built upon findings and methods from the DOVE project – including the feedback received when tailoring presentations to policy priorities – and created the Vaccine Economics Research for Sustainability & Equity (VERSE) Toolkit. This toolkit runs multidimensional equity assessments on existing data, demonstrating the impact of vaccines comparable with other sectors. Equity in vaccination coverage is mainly conceptualized using wealth-based equity measures such as those employed by USAID’s EquityTool and WHO’s Health Equity Assessment Toolkit (HEAT). Applying the VERSE Equity Toolkit to Demographic & Health Surveys and local datasets, evidence suggests tools substantially underestimate the magnitude of inequity in vaccine coverage. Wealth-based inequality measures systematically underestimate the gap between most and least advantaged in fully immunized coverage, by 0.2 – 34.9 percentage points globally, correlated with maternal education, geography, and sex. Closing the gap between the bottom and top wealth quintiles is unlikely to eliminate persistent socio-demographic inequities in both coverage and access to vaccines. Based on these results, pro-poor interventions and programs utilizing needs-based targeting should expand their criteria to include other dimensions to holistically reduce systemic inequalities.

The Initiative

The VERSE project developed a toolkit that uses both existing and novel methodologies to rank a country’s population by level of composite disadvantage, composed of known factors that fairly and unfairly influence access to and uptake of vaccines. For routine pediatric vaccines, fair factors include whether the child is too young to receive a vaccine based on the national immunization schedule. Unfair factors include the household’s region, setting (urban/rural), and socioeconomic status (SES), as well as the sex of the child, level of maternal education, and whether the household is covered by any health insurance. These unfair factors are used to generate a propensity score for disadvantage, allowing households to be ranked from most disadvantaged to most privileged in terms of accessing vaccines. These needs-adjusted and multivariate ranking criteria form the bases for generating several equity metrics, including a concentration index and an absolute equity gap which can be used as a measure of overall vaccination coverage inequity in a health system. Unlike the VERSE Equity Toolkit, most existing equity metrics do not appropriately account for the intersectionality or compounding effect of geographic and sociodemographic factors on healthcare access and outcomes. Furthermore, different fair and unfair parameters can be included based on the needs of partners and policymakers.

VERSE is partnered with Makerere, icddr,b, and their network of public agents and policymakers. Once the toolkit was released in open access, the team was approached by Dr. Hai Fang at Peking University to discuss its application to non-public data provided by the Chinese Centers for Disease Control. Other partners in India, Ethiopia, and Ecuador have approached since then. A critical challenge VERSE tackles is translating outputs to specific policy solutions and interventions, as there are numerous ways solutions could be designed and it is not a prescriptive approach.

Engagement

The VERSE Equity approach includes three steps to engage partners, governments, and expert representatives. First, policymakers, public agents and local experts identify health outcomes relevant to monitor and evaluate (M&E) a policy, program, or priority area derived from existing M&E reports. Guided by our partners, they also brainstorm factors related to need and eligibility (“fair” factors) and factors that should not affect the outcome of interest (“unfair” factors) to build a multivariate ranking scheme. Second, analysts - either public agents or members in our partners’ teams - customize the Toolkit’s program code and set up the equity assessment with the chosen outcome and fair/unfair factors. Third, analysts run the program and organize the outputs for the initial group of policymakers to review and interpret. Throughout this process, experts and decisionmakers are engaged to develop the composite equity approach and to identify the appropriate data to run analysis.

In October 2022, after months of engagement with its government and NGO partners, Makerere University hosted an intensive 2 week seminar and invited its partners to a series of brainstorming sessions and R programming workshops. As a result, their partners used the toolkit to generate and interpret vaccine coverage and equity outputs for Uganda from the latest DHS, presenting findings through virtual posters. Several new research and programmatic questions were identified during the sessions. A group struggling to meet a sponsor’s expectations to address health equity in their M&E reports set up an analytical design that would help them fulfill this difficult task with their data. The approach to equity and the algorithm in the toolkit were developed with an advisory group composed of equity-focused experts and representatives from multilateral agencies (WHO, Gavi), academic institutions, and specialists from thinktanks and NGOs. The group’s involvement ensured the standardization of indicators and outputs and highlighted how organizations on the receiving side of government and M&E reports would view and use such metrics.

Impact

The VERSE Equity Tool only officially launched recently. The impact of the tool’s outputs on policy and decision-making is being observed as our partners complete analyses and begin translating the results to stakeholders. In Uganda, the partnership includes advanced teams from the Ministry of Health, Expanded Programme on Immunization, the Coalition for Health Promotion and Social Development, and the Uganda Bureau of Statistics. These partners have attended training and are undergoing applications of the VERSE toolkit to respond to specific policy questions surrounding the unmet need for family planning, using administrative data to track vaccine equity and equity in access and outcomes of antiretroviral therapy for HIV/AIDS viral suppression. The results will inform specific decisions and be utilized to demonstrate changes in equity, as well as the impact of specific programs on equity trends.

In China, the results of VERSE were published in JAMA Network Open and will be presented to the Chinese CDC to inform their decisions on which non-immunization program vaccinations to incorporate into the national immunization program. In Ecuador, results from the VERSE model were used to inform an investment case for the pneumococcal vaccine and were presented to the ministry of health and finance, resulting in continued financing for the vaccine. The results showed the geographic distribution of benefits from vaccination - within Ecuador and nearby regions, the vaccine had significant positive health and economic impact over time.