Numbers that mislead
"Doubles your risk" from 1 in 1,000 to 2 in 1,000, a chart axis that starts at 50, and ice cream that "causes" drowning: the light-touch statistics tricks.
FROM WHAT TO WHAT?
Relative risk versus absolute risk
"Doubles your risk" is a relative change: how the new risk compares to the old one. The absolute change is how many more people are actually affected. If a risk goes from 1 in 1,000 to 2 in 1,000, it has doubled (a "100% increase"), yet it means one extra case per 1,000 people. Both statements are true. Only one lets you judge how much it matters to you.
A headline says a food "raises your risk by 50%". From 2 in 100 to 3 in 100 is a big deal. From 2 in 100,000 to 3 in 100,000 almost certainly isn't. The headline alone can't tell you which.
Headline: "This habit DOUBLES your risk"
Without the habit: 1 in 1,000 (0.1%)
With the habit: 2 in 1,000 (0.2%)Relative change: 2 / 1 = double, a "100% increase"
Absolute change: 1 extra case per 1,000 people
= 0.1 percentage pointsEven with the habit, 998 of 1,000 people never get it. Always ask "out of how many?" and "from what to what?"
Check yourself
A report says a side effect's risk "doubled", from 2 in 10,000 people to 4 in 10,000. How many extra people per 10,000 are affected?
- 2
- 4
- 100
- 5,000
Show the answer
2
Right. 4 minus 2 is 2 extra people per 10,000. The risk did double, and it's still small in absolute terms.
Percentage points
The plain difference between two percentages. If an interest rate goes from 10% to 12%, it rose by 2 percentage points, which is also a rise of 20% (2 is a fifth of 10). Both are true. Mixing them up is an easy way to make a change sound bigger or smaller than it is.
"Support rose 50%" could mean from 40% to 60% of people, or from 2% to 3%. "Rose 1 percentage point" leaves no room for that confusion.
Check yourself
Unemployment falls from 10% to 8%. Which description is accurate?
- It fell by 2%
- It fell by 2 percentage points, which is a 20% fall
- It fell by 20 percentage points
- It fell by 8%
Show the answer
It fell by 2 percentage points, which is a 20% fall
Yes. 10 minus 8 is 2 percentage points, and 2 is a fifth of 10, so it's also a 20% fall. A careful report says which one it means.
THE MISSING PART
Missing denominators and tiny samples
"500 cases in our city" sounds alarming, but it means little until you know out of how many people and how many is normal. In a city of 5 million that sees 450 cases in a usual month, it's barely news. The same goes for surveys: "80% of users loved it" might be 4 people out of 5. Small samples swing wildly by chance.
A precise-looking number ("73.4% of experts agree") can feel more trustworthy than a round one. Precision is not accuracy: ask who was counted and how many.
Check yourself
An app advert says "80% of users recommend us". That tells you as much whether the survey asked 5 users or 5,000.
Show the answer
False
False. With 5 users, 80% is 4 people, and one more or one fewer changes the result by 20 points. With 5,000 people, chance evens out and the figure means much more. Always ask "80% of how many?"
Same numbers, two charts
Axis starts at 50
Two bars show 52 and 55. Measured from 50, one bar is 2 units tall and the other 5, so the second looks more than twice as big. The gap looks dramatic.
Axis starts at 0
The same 52 and 55, measured from 0. The bars are almost the same height, because the real difference is about 6%. Nothing dramatic happened.
Check yourself
A bar chart compares two phone brands' satisfaction scores: 92 and 96. The y-axis starts at 90, so one bar looks three times taller than the other. What's going on?
- The second brand really is three times better
- The truncated axis exaggerates a small difference of about 4%
- The chart is fine, because the numbers on it are correct
- The chart should start at 100 to be fair
Show the answer
The truncated axis exaggerates a small difference of about 4%
Right. Measured from 90, the bars are 2 and 6 units tall. Measured from 0, 92 and 96 are nearly the same height. The numbers are true; the drawing misleads.
TOGETHER IS NOT BECAUSE
Correlation isn't causation
Ice cream sales and drownings rise and fall together. Ice cream doesn't cause drowning: hot weather drives both, since people buy ice cream and go swimming. A hidden third factor like that is called a confounder. Whenever two things move together, ask three questions: Could a third thing cause both? Could it be the other way round? Could it be chance?
Collections of "spurious correlations" match unrelated yearly numbers that happen to rise and fall together, such as a country's cheese eating and some unrelated accident count. With enough numbers to compare, some will line up by pure luck.
Check yourself
Data show that fires with more firefighters at the scene cause more damage. What's the most likely explanation?
- Firefighters make fires worse, so fewer should be sent
- A third factor: bigger fires bring both more firefighters and more damage
- It's pure coincidence and means nothing
- Damage attracts firefighters after the fire is out
Show the answer
A third factor: bigger fires bring both more firefighters and more damage
Yes. The size of the fire is the confounder: a big fire does more damage and gets more crews. Sending fewer firefighters would make things worse, not better.
Ten people's yearly income, in units:
9 people: 10 million each
1 person: 1,000 million (1 billion)Mean (add up, divide by 10):
(9 × 10 + 1,000) / 10 = 109 million
Median (the middle value when sorted):
10 million
9 of 10 people earn far less than the "average".One very large value drags the mean up; the median barely moves. When a report says "the average salary", ask which average. For income, house prices and wealth, the median usually describes the typical person better.
Check yourself
Which question would you ask first about each claim?
- "Cases of the illness doubled this year"
- "Children who eat breakfast get better grades, so breakfast makes kids smarter"
- "9 out of 10 customers were happy"
- "People who own a dog live longer, so get a dog"
- "This habit raises your risk by 40%"
- "Cities with more hospitals have more deaths"
Show the answer
Out of how many?: "Cases of the illness doubled this year", "9 out of 10 customers were happy", "This habit raises your risk by 40%"
What else could cause it?: "Children who eat breakfast get better grades, so breakfast makes kids smarter", "People who own a dog live longer, so get a dog", "Cities with more hospitals have more deaths"
Lesson recap
- "Doubled" is a relative change. Ask for the absolute one: from 1 in 1,000 to 2 in 1,000 is one extra case per 1,000.
- From 10% to 12% is 2 percentage points, or a 20% rise. Check which one is meant.
- A number needs its denominator and its normal level; "80%" of 5 people is 4 people.
- Check where a chart's axis starts, which time range was chosen and whether the steps are even.
- Things that move together may share a hidden cause, run the other way round, or match by chance.
- A few huge values pull the mean up; the median often describes the typical person better. Base rates are covered in the Mental Models course.