Every day, we get an update on Covid-19 cases and deaths. Or we hear about police having confiscated this many items of contraband. Without context, such numbers are meaningless at best, and may be intended to deceive.
One of my biggest gripes with media reporting is the naïve treatment of numbers. It’s as if journalists can’t do basic arithmetic, can’t visualise numbers in their head, or couldn’t be bothered trying to think about what numbers mean.
Whenever you come across a big number, ask yourself, ‘Is that a lot, or not very much at all?’
There are many ways to reach an answer, but countless newspaper articles don’t give you the information to contextualise the numbers they trumpet in their headlines. Often, the numbers serve the interests not of the reader, but of those whose press releases the media uncritically parrots.
Daily Covid-19 numbers
Every day, we get a Covid-19 update. For most, it will come in the form of a newspaper headline, such as this: ‘South Africa records 144 new Covid-19-related fatalities as death toll climbs to 12 987’.
The article itself is not much more informative. It adds that there are now 607 045 confirmed cases, and 504 127 recoveries, for a recovery rate of 83%. It then breaks down the case numbers by province.
All this data is totally useless. What readers want to know is how bad things really are, and whether things are getting better or worse. None of these numbers, in isolation, answer any of these questions.
Alex Welte, writing for GroundUp, does a great job explaining why the recovery rate does not mean what you might think it means, and is a ‘lousy measure of how we’re doing’.
As the epidemic rolls on, a greater number of all the cases ever recorded get resolved, so it stands to reason that the recovery rate will gradually increase, the longer the epidemic lasts.
He also notes that the underlying numbers are questionable. We don’t really know how many Covid-19 deaths there have been, as some number undoubtedly happened in circumstances where they could not be counted.
We also aren’t sure about the infection numbers. We don’t know how many people with no or few symptoms present themselves for testing. Even among people who do get tested, we have the problem that perhaps 30% of results could be false positives, and many more could be false negatives, especially if the tests are conducted in the early days of the infection. Even at the optimum of eight days after infection, 20% of tests return false negatives.
If we’re not sure about the real number of cases, and the real number of deaths, we can’t possibly rely on a composite statistic such as the case-fatality ratio (CFR) to give us a better idea of where we’re at. If you divide garbage by garbage, you get worse garbage.
Where are the charts?
What we should do, is look at trends. Are things getting worse, or are they getting better?
Newspapers should be publishing charts, not daily numbers. Government should be publishing charts, not daily numbers. We know that the country is over the peak, but sadly, charts that indicate trends in infections or deaths are few and far between (or behind paywalls so only a handful of rich people get a look at them).
Even the charts that the National Institute for Communicable Diseases (NICD) publishes are not particularly useful. They show cumulative cases and deaths, which suggests a flattening of the curve, but not daily cases and deaths, which would confirm that suggestion.
The NICD’s age distribution chart is meaningless because it is not normalised. We know we have few old people, so we shouldn’t be surprised there are few cases among old people, and that the curve skews much younger than elsewhere in the world. The question is how the chart would look if you divided cases or deaths in an age category by population in that age category.
Much is made of the fact that South Africa ranks fifth in the world in terms of the number of cases, but we’re only 21st in terms of number of cases per million population. We rank 13th in terms of total deaths, but only 29th in terms of deaths per million population.
Why aren’t numbers reported per capita? It is of no use to know that we have more cases or deaths than some small country with five million people. To make a comparison, we must have per capita numbers.
Why is it that everyone knows the ‘fifth in the world’ factoid, but few know that per capita, our death rate is not at all extraordinary? Why does everyone know about 13 000 Covid-19 deaths, but nobody knows that influenza kills 11 000 every year, and this year, there appears to be no influenza at all?
Without context, daily Covid-19 numbers are meaningless. They’re a waste of valuable screen time that could more profitably be spent arguing with bigots on Facebook.
Busting smugglers
Or take this article: ‘SARS has confiscated illegal cigarettes worth R77m during lockdown, says Mboweni’.
Mboweni placed this number into some sort of context by saying this compares to R15 million during the same period last year.
Now, is R77 million a lot? Does quintupling the haul of contraband mean the taxman has done a good job cracking down on the black market?
Let’s leave aside that there is no way to tell the value of a given haul of cigarettes when there is no legal market. Were the confiscated cigarettes valued at pre-lockdown prices, or at black market prices? Black market prices varied widely by location and over time. (For the same reason, never trust a value that police place on a drug bust. They have no clue what the true street value of a big haul is.)
There is no good estimate of the total size of the black market in cigarettes, but we can piece together some estimates to put that R77 million into context.
The legal South African tobacco market, in 2017, was valued at $2,7 billion, or about R45,7 billion at today’s exchange rate. All cigarettes were illicit during the lockdown, and the ban lasted about four months. About 90% of the country’s 11 million smokers continued to smoke during lockdown. Illicit cigarette prices were on average 250% higher than pre-lockdown prices.
That leaves us with a total estimated black market size of as much as R34 billion, or, if we assume smokers cut down by 50% to compensate for higher black market prices, some R17 billion.
That means R77 million represents less than half a percent of the total illicit cigarette market during lockdown. Those are pretty decent odds for black market dealers.
How about last year’s number, then? Well, before the lockdown, the illegal trade accounted for between a third and half of all cigarettes sold. Using the conservative estimate, that means SARS confiscated about 0.3% worth of the illicit market.
So although SARS was marginally better at finding illegal cigarettes when all cigarettes were illegal, it was not particularly effective in curbing the illegal tobacco trade during lockdown.
A better headline would have read: Mboweni admits SARS failed to seize more than 99.5% of illegal cigarettes during lockdown.
Numbers need context
There are countless examples of contextless numbers that are presented as somehow having meaning.
Financial amounts involved in corruption cases. Kilograms of CO2 emitted by this or that activity. Acres or hectares of land affected by a fire. Number of views on YouTube. Stock prices. Project spending.
If a news article says that a company is building a R100 million facility, is that a lot? Or are they getting value for money? If it says Eskom is building a R150 billion power station, is that much? Should they cost a lot less?
Unless you’re an expert in the field, you simply can’t know. Numbers in news reports would be far more informative with added context. Compare them to the past to show trends. Compare them to similar events to gauge whether they’re high or low. Compare them to totals to get a sense of scale. Use averages. Normalise data.
Without context, numbers can convey a lot of things – irrational fear, undeserved praise, misguided outrage – but they cannot convey useful, actionable information. Often, those who produce those numbers are counting on that.
[Picture: Gerd Altmann from Pixabay]
The views of the writer are not necessarily the views of the Daily Friend or the IRR
If you like what you have just read, subscribe to the Daily Friend