Evaluation of Gavi's Fragility, Emergencies and Refugees policy

Evaluation objectives

Gavi’s 2017 FER Policy aims to allow flexibility in applying Gavi policies and processes in countries that face fragility, emergencies or a refugee situation. The evaluation assessed the overall design, implementation and results of the policy. The main objectives of this evaluation are:

To assess the extent to which the design of the policy was relevant and appropriate to achieve its intended purpose and objectives.
To assess the effectiveness and efficiency of the implementation and management of the policy by the Gavi Secretariat.
To assess the extent to which the policy has achieved its desired result(s). Evaluators should describe the main successes, challenges and lessons learned.
To provide evidence-based findings and recommendations to assist in the review and update of the Policy and inform the fragility segment.

Methods

The evaluation is retrospective in nature, covering the period from when the FER policy was designed and first applied (i.e., from June 2017) until mid-2020 and covers the application of the FER policy for non-COVID-19-related fragility and emergencies. It primarily focuses on desk review, key informant virtual interviews, and country case studies (Afghanistan, Bangladesh, and South Sudan). The evaluation is not intended to cover Gavi’s approach/response to the COVID-19 pandemic.

Findings and conclusions

The evaluation findings show that the FER policy is considered relevant but could improve by addressing eligibility criteria and rapid operational response requirements. While the policy is aligned with global response guidance in humanitarian situations, alignment with the Gavi Strategic Period 5.0 is recommended, underscoring the need to address pockets of low immunisation coverage in fragile settings and focusing on emergencies and refugees.

Relevance

The FER policy is relevant for Gavi’s direct and indirect beneficiaries and robust enough to serve its primary function as an instrument to allow flexibility in applying Gavi policies and operations in countries that face fragility, emergencies, or a refugee situation. The policy clearly articulates the eligibility criteria by making use of internationally published lists for defining fragile countries and drawing on the classification used by the World Health Organization (WHO) and the United Nations Office for the Coordination of Humanitarian Affairs (OCHA) for emergency situations. However, the relevance of the policy is challenged in FER situations that require agile responses and that demand rapid adaptation of programmatic, financial and administrative approaches. The relevance of the policy is also challenged in Gavi-eligible countries that face localised fragility but which are not eligible for support according to the criteria of the FER policy, and in non-Gavi-eligible countries that face fragility or an influx of refugees.

The FER policy aligns well with global guidance and immediate responses to humanitarian situations.

Effectiveness and efficiency

The Operational Guidelines are clear and provide a sufficient basis for ensuring consistency in the application of the FER policy but this has not prevented differing levels of understanding and varying interpretations of the policy, both internally and externally, in part because the guidelines are only an internal document for Secretariat staff, and cannot be accessed externally.
Gavi has been increasingly successful in joining country-level humanitarian coordination platforms. Pooled funding mechanisms that resource joint programmes provide Gavi with access to key non-government partners that can cover hard-to-reach, conflict-affected, and opposition-controlled areas. The FER policy has enabled Gavi to seek this coherence and effectiveness in its programmatic approach.

Intended results

The intended result of the FER policy, increasing the effectiveness of support towards equitable access to immunisation, was achieved, according to a large majority of informants, but this could not be fully assessed by the evaluation team, for the following reasons:
1. data are not captured and centrally consolidated and analysed by Gavi for decision-making, due to flexibility outcome data being either unavailable or barely available although an adequately designed M&E framework does exist
2. National annual routine immunisation data available through country Grant Performance Frameworks (GPF) do not allow for the analysis of specific populations, targeted areas, refugees, migrants, IDPs, or emergencies. The evaluators found mixed evidence on the progress of the coverage of selected indicators (Penta, IPV, MCV, etc.) at national level.
The country case studies show that the requirement for using quality data and analysis (which are often not readily available) hampers quick responses and decision-making, and leads to lengthy negotiation and approval processes.

For the evaluation recommendations and Gavi Alliance management response please see the full report and Evaluation Management Response.