Back to
Our analysis shows a significantly different picture than the one presented in a study by the Institute for Health Metrics and Evaluation (IHME) issued in December.
- Differently than the IHME study, we have identified only eight countries who may have received "overpayments" of ISS rewards, amounting to between US$ 30 million and US$ 40 million -- an average of just over 1% of GAVI's total disbursements to date. The majority of this money, approximately US$ 33 million, was authorised for payment to one country but only half has been paid to date.
- We have found no evidence of misappropriation of funds. Nevertheless, GAVI will now look further into these countries by reviewing decisions by the Independent Review Committee or IRC (which approves payments), examining additional data, and visiting a number of the countries themselves.
- Our analysis is not yet complete. GAVI is working with WHO to assess four additional countries.
- GAVI is a risk-taking organisation. When ISS was conceived, Alliance partners consciously took the decision to give the responsibility for data to the governments whilst recognising the variable quality of country data systems and the significant differences between household survey and administrative data in some countries. This was viewed at that time as essential to strengthening country systems and ownership of them rather than creating parallel systems.
Differences in ISS data
The IHME research is based upon modelling and data from household surveys which is different from the data GAVI uses. Under the current arrangements, recommendations for ISS rewards by the Monitoring Independent Review Committee (IRC) are based upon the administrative data reported in the countries' Joint Reporting Form. This data is reviewed by WHO and UNICEF before being forwarded to the IRC. Separately, WHO and UNICEF review data from a variety of sources to estimate likely immunisation coverage and provide their own adjusted estimates retrospectively.
GAVI-WHO analysis
WHO and UNICEF firmly stand behind their coverage estimates and GAVI uses these estimates to measure immunisation progress. For the period of 2002-2006, we verified that 35 of the 51 ISS countries either had administrative data that were similar to the WHO/UNICEF estimates, did not receive ISS rewards, or were within the confidence intervals of the IHME study estimates. (Confidence intervals provide mathematically produced estimates of uncertainty, which are necessary when dealing with data drawn from incomplete samples.) Three additional countries received lower reward payments based on their administrative data than they would have received based on the WHO/UNICEF estimates, about US$ 1.5 million. These countries may have been eligible for higher reward payments. In another country, the IRC approved a reward payment in 2006 based upon country data that was consistent with the WHO/UNICEF estimates at the time. However, the estimate was later revised downwards.
Possible overpayments much lower
In eight of the remaining 12 countries on the list, we found that administrative data was higher than WHO/UNICEF estimates. Although the ISS was not designed for reward payments to be based upon WHO/UNICEF estimates, we compared reward payments that would have been made using these estimates to payments we made using administrative data. From this comparison, we can deduce that the divergences between administrative data and the WHO/UNICEF estimates produced a possible "overpayment" to these eight countries of between US$ 30 million and US$ 40 million, significantly less than then stated by IHME.
IHME analysis
The IHME estimation of overpayment is problematic. Firstly, it is based on a greater number of countries than should be included. (Indeed, two countries that were included did not at any time receive reward payments). Also, the research group lumped together the grant disbursements from the investment phase and the payments from the reward phase. Finally, they relied on modelled data with extensive missing values and did not take into account uncertainty in the statistical models. The IHME research does confirm some issues, for which the Data Task Team was established to look at. In particular, the team has been asked to examine why there is a difference between WHO/UNICEF estimates that we currently rely on and the data generated in the IHME study. Once the team's recommendations are completed in March, GAVI will determine how best to improve ISS.