Is There Any Merit in Objections to WHO's Excess Death Estimates?

Academic studies published since last quarter of 2021 began to peg India's excess mortality at between 3.5 and 6 million deaths for the two waves. Finally, the World Health Organization published estimates of pandemic-associated excess mortality in early May 2022, placing the COVID-related toll in India at roughly 4.7 million for 2020 and 2021.

The Indian government, and some scientists and journalists, have pushed back against these claims, arguing that they are not based on accurate data and do not correctly reflect what is known about mortality in India.

In response, many have accused them of blindly supporting the government, without fully considering what some of the objections are. FactChecker examines some of these claims and shows what the data really says.

'Aimed to Malign India's Image'

Claim 1: While rejecting the WHO's report, NITI Aayog Member (Health) Vinod K Paul said, "These kinds of assumptions used for a nation of India's size to put us in a poor light is not desirable"

In another instance of the data being called an attempt to malign India's global image, India Today quoted sources from the Health Ministry as saying that WHO's excess mortality figures are a "vicious agenda to malign a country emerging as a credible healthcare hub in the pandemic".

Fact: Countries around the world have missed recording some COVID-19 deaths for a range of factors, and estimates of excess mortality help them approach estimates of the true toll.

The WHO defines pandemic excess mortality as "the difference between the total number of deaths estimated for a specific place and given time period and the number that would have been expected in the absence of the pandemic (no-COVID-19 scenario). This difference is assumed to include deaths attributable directly to COVID-19 as well as deaths indirectly associated with COVID-19 through impacts on health systems and society".

Countries around the world accept that they missed some deaths from COVID. Official COVID counts "miss those who died without testing, they are contingent on the country correctly defining COVID as the cause-of-death and they miss the increases in other deaths that are related to the pandemic leading to overwhelmed health systems or patients avoiding care" the WHO says.

Additionally, some countries or regions may experience lower than expected mortality particularly during lockdown as a result of reduced infectious disease related mortality as well as reduced transport and injury-related fatalities.

The United Kingdom, South Africa and Malaysia are among the countries whose government or government-funded research bodies have produced their own national estimates of excess mortality during the pandemic by monitoring total deaths during and before the pandemic.

In February 2021, the WHO began the process to bring out COVID-related excess mortality estimates for the world. India is just one of WHO's 194 member countries for which the organisation has produced estimates.

The WHO estimates fit with the numbers suggested by at least three other studies. These studies use a combination of the dataset used by the WHO as well as other datasets and surveys to triangulate findings.

While a report by Jha et al., published in January 2022, estimated excess mortality at 3.2 million during June 2020-July 2021, a working paper published the same month pegged the excess deaths at 6.3 million for a longer period — January 2020 to August 2021. One more study, published half a year before these two, estimated the excess deaths from 18 months of the COVID-19 pandemic (January 2020-June 2021) to be between 3.4 million and 4.9 million.

'No Data, Just Modelling'

Claim 2: The Centre objected to the WHO using mathematical modelling and said the global agency did not use any actual data for India.

"The analysis [WHO's estimates] while uses mortality figures directly obtained from Tier –I set of countries, uses a mathematical modelling process for Tier II countries (which includes India)," read the Ministry of Health and Family Welfare's statement published in response to an article by the New York Times on the wrangling between the WHO and the government.

Fact: The WHO used data from 18 Indian states and Union Territories for which data was available, and extrapolated this for the country.

"Pure modelling was required for countries for which no data was available. However, this was not the case for India," Dr Samira Asma, Assistant Director-General for Data, Analytics and Delivery at the WHO, clarified to FactChecker.

The WHO's excess mortality estimates classified countries into two tiers. Tier 1 countries included those that provided complete and nationally representative monthly all-cause mortality data for the specified period to the WHO. Tier 2 countries, in which India was categorised, included countries whose complete data WHO did not have access to.

In India's case, no mortality data for 2020 or 2021 was made available to the WHO, the global health agency told FactChecker. Then, 48 hours before the WHO's estimates were released, the Indian government produced national data for 2020, which was, however, not disaggregated by month.

However Indian journalists had accessed official monthly mortality data for 18 states and UTs and made this data publicly available on a free and open portal, following which several researchers used this data to write academic papers. Owing to the inaccessibility to any data, the WHO used these numbers (which covered 70% of the country's population) and extrapolated them for the rest of the country using a documented methodology.

Now that the WHO does have some Indian data for 2020, it will study the data, Dr Asma said.

'Govt's Data Don't Match WHO's'

Claim 3: The central health ministry claimed that mortality data produced by the Indian government for 2020 do not match those used by the WHO. "The Civil Registration System data of 2020 published by Registrar General of India on May 3, 2022 clearly reveals that the narrative sought to be created based on various modelling estimates of India's Covid-19 deaths being many times the reported figure is totally removed from reality," claimed the ministry in its May 5, 2022 statement.

Fact: Yes, there are differences but they are unlikely to affect aggregates significantly.

There are indeed differences at the state level in the data produced by the Indian government for 2020, and the data gathered by journalists. However, while the absolute numbers might seem some distance apart, the difference between 2020 and 2019 — the excess mortality in essence — is small. For 12 large states for which journalists had gathered data, the excess mortality derived from the numbers collated by them and hence used by the WHO differ by around 27,000 deaths, or a difference of just 5%.

Moreover, the WHO estimates suggest that the majority of pandemic mortality (over 80% of total excess deaths in India) took place in 2021, not 2020, for which the India government has made no data available.

However, a clear idea of what the differences at the level of each state are and why this is so would be useful.

'Excess Deaths Due to More Registrations'

Claim 4: NITI Aayog's VK Paul claimed that the increase in deaths was on account of an increase in registration of deaths, not actual deaths. He claimed, "More death registration is also happening because people are conscious, that they need death certificates for property and other purposes. And generally also due to ease of operation and digitisation, people are coming forward. Population size also increases every year, contributing to more deaths."

He also highlighted that excess death registration increase has been noted in previous years despite a declining death rate and no outbreaks. "So, we must remember that extra deaths are not due to COVID-19 deaths, but there are other reasons and causes as well," added Paul.

Fact: It isn't evident that there was an improvement in the registration of deaths. On the contrary, there is some evidence that death registration has fallen.

Before the pandemic, the registration of deaths in India was improving. However, there is no evidence yet that registration improved in 2020 and 2021, despite the Indian government's most recent claim that death registration in 2020 was 99.9%, this number is unsubstantiated. Moreover, taking this unsubstantiated claim at face value would result in the implication that states like Uttar Pradesh experienced huge unexplained declines in deaths in 2020. The fact is that the level of death registration has in the past been derived from the Sample Registration Survey, which was not conducted in 2020.

Meanwhile, data from the 2019-20 National Family Health Survey indicates that death registration could have fallen in 2020, meaning that increases in mortality would not be a 'registration effect'.

How much of the increase in mortality during the pandemic should have been expected and how much was "excess" will always be a result of assumptions, whether by the WHO or by the government, because India does not yet have a completely reliable baseline mortality number.

Even prior to the pandemic, the Indian CRS was not able to capture every death; one study estimates that the CRS missed 2.28 million Indian deaths in 2019, of which more than half were deaths among the elderly. This could have had an impact on the number of deaths missed during the pandemic - not only were the elderly particularly vulnerable during the pandemic, one study in the city of Madurai also showed that deaths among the elderly were more likely to have gone uncounted in the official COVID death toll.

The WHO uses the Indian database for 2020 and produces estimates using its stated assumptions to compare pandemic mortality with what it would expect baseline mortality to look like based on its own Global Health Estimates data. While the Indian government has not objected to the WHO's GHE numbers in the past, it now disputes these estimates on unspecified grounds.

Finally, the vast majority of deaths in 2021 took place over just two months, when an increase in registration is unlikely to have had much of a role to play.

'Relief Claims Much Lower'

Claim 5: Dr Randeep Guleria, Director of the All India Institute Of Medical Sciences, claimed that since the number of compensation claims do not run into millions there could not be that many missed deaths. "Even if there were excessive deaths that were COVID related, they would have been recorded because people would have come forward, and their relatives would have come forward for compensation. This has not been the case as far as the numbers that WHO is predicting," said the AIIMS Director.

Fact: The number of claims is unclear, there are barriers to making claims, and data on some claims show an increase.

The precise number of claims of COVID compensation made thus far is hard to ascertain. The Indian Express reported that around 6.87 lakh claims for COVID death compensation were made in 11 states. Other sources suggested that by February 2022, states had received nearly 7.5 lakh claims. However, given that a claim required at the minimum a positive test for COVID, it is likely that deaths were missed. Moreover, there were multiple media reports on the barriers that a lack of paperwork created for people trying to access COVID compensation.

Additionally, while the government was prepared to point to this data, it was quick to reject another source of data — data from the Life Insurance Corporation which showed a large increase in death claims in 2021. While the government's clarification did point out that not all the deaths could be attributed to COVID, it failed to address the sudden overall increase in reported claims.

Finally, the WHO's estimates are not of COVID deaths alone, they are for deaths both directly and indirectly associated with the pandemic.

FactChecker had asked the ministry's spokesperson for clarification on their above-mentioned claims via email, but had not received a response by the time of publishing this article. If and when we do, it will be updated here.

(Nileena Suresh, data journalist at IndiaSpend, contributed to this story.)