Findings and conclusions

• CCEOP has achieved its goal in increasing the number of optimal CCE across Gavi countries, contributing to increased immunization services, a more reliable supply chain, and improved vaccine potency.
• Processes established, such as the Project Management Team (PMT) and Service Bundle Provider’s (SBP) reporting requirements, have been effective at strengthening management capacity and country ownership. PMTs have demonstrated leadership in coordinating across donors and partners for supply chain strengthening and CCE procurement.
• While CCEOP has improved the CCE availability, this evaluation has identified other areas of the immunization supply chain that need to be strengthened to ensure this new equipment continues functioning and that the ultimate goal of improving vaccine coverage and equity is reached. It is evident that CCE maintenance systems need investment and re-visioning to be more effective.
• Many of the processes and reporting requirements established for CCEOP can be adapted and continued for ongoing monitoring of the CCE and the immunization supply chain in general to reiterate best practices and guidance.
• It is clear that lines of communication between all stakeholders — between health system levels and between country-level and global-level decision makers — can be improved. While a complete institutional analysis was out of scope for this evaluation, the results do imply that many gaps in implementation, such as the lines of communication, reflect the system reality rather than the CCEOP structure.

For the evaluation recommendations and Gavi Alliance management response please see the full report and Evaluation Management Response.

• Endline Evaluation of Gavi’s CCEOP - EAC Final Assessment

  Download

FINDINGS AND CONCLUSIONS

• CCEOP has been successful in stimulating a market for - and catalyzing awareness, availability, and use of - higher performing Cold Chain Equipment (CCE) in many countries. There is value in more organized and aggregated procurement and information sharing for CCE.
• The original CCEOP Market Shaping (MS) strategy highlighted some of the potential challenges to achieving the MS objectives that ultimately played out, but was insufficiently robust to address them. The revised MS strategy appears to address some of those limitations, and progress has been made in promoting a healthier market and shifting demand to some of the lesser utilized CCE suppliers.
• The revised MS objectives were appropriate to address some of the challenges observed with the original objectives, which at times were at odds or mutually inconsistent. However, the revised objectives have not fully reconciled the tensions between observing country preferences, achieving continuous product innovation, recognizing cost savings, and maintaining a relatively large number of interested suppliers - alongside a relatively static demand outlook.
• The supply-side objectives of CCEOP have been met, supported by a strong base of platform eligible suppliers and range of options across iced line refrigerators and solar direct drives and size segments.
• Deliberate MS efforts have resulted in a third supplier accumulating greater market share in 2019 and alleviating concerns around the duopoly observed in the early years of CCEOP. However, procurement volumes will need to remain on track and maintain or expand market split through 2020 and beyond to continue to meet MS goals.
• The mandated service bundle component created initial complexity for CCEOP, and MS efforts in particular, but as all of the parties involved became more familiar with implementation, valuable lessons have been learned about the benefits as well as opportunities to be more flexible with service bundle options in different types of country contexts.

For the evaluation recommendations and Gavi Alliance management response please see the full report and Evaluation Management Response.

FINDINGS AND CONCLUSIONS

• CCEOP is responsive to country needs and priorities and is well coordinated by the Project Management Team (PMT) in-country with other donors and partners.
• Not all implementation guidance, details of warranties, policies or plans are well communicated to stakeholders at all levels in the country. As a result, they were not consistently understood or implemented at the country level.
• Stakeholders were generally satisfied with the Service Bundle Provider’s (SBP) installation of equipment and recognized the trade-off between the speed and effectiveness of SBPs deploying CCE and value for money considerations.
• The SBP monitoring system for deployment was effective and allowed for minor deviations.
• SBPs provided training to health facility staff also communicating maintenance and repair protocols. Training was also provided at the national level. Yet, the training was found to be insufficient with the need to clarify training needs and topics.

For the evaluation recommendations and Gavi Alliance management response please see the full report and Evaluation Management Response.

• Midline assessment summary of Evaluation of the Cold Chain Equipment Optimization Platform (2020)

  Download

• Cold Chain Equipment Optimization Platform: Market Shaping Policy Brief

  Download

FINDINGS AND CONCLUSIONS

• The CCEOP application was based on country priorities, using information from available data through information systems and from cold chain inventory.
• The Project Management Team (PMT) plays an active role in CCEOP deployment and coordination.
• High level of stakeholder engagement but limited role beyond the national level.
• There were mixed views on the complexity of the application process.
• Review of the operational development plan (ODP) and the CCEOP applications in all countries indicates a high level of alignment with Gavi guidelines and fit with country strategies.
• A general concern in all countries is that health workers may face difficulty following equipment manuals and handling the equipment.
• There were mixed feelings and confusion about the Service Bundle Provider (SBP) approach, especially in lower levels of the health system.
• Although CCEOP equipment has not been fully deployed, stakeholders anticipate several positive outcomes—an improvement in the overall efficiency of the system, a reduction in the need for corrective maintenance, and eventually improvement in immunization coverage.
• CCEOP is effective in addressing the CCE fundamental of Gavi’s immunization supply chain strategy; however, the overall system must also be strengthened in terms of, for example, a maintenance plan that is funded and is functional, human resource knowledge and skills on temperature monitoring and vaccine management practices, and a distribution system that is effective.

For the evaluation recommendations please see the full report.