Most People Don't Have a Clue About Statistics
Contents
We live in a world run on numbers. Medicine, economics, crime, elections, public policy: almost every claim that matters is a statistical claim. And most people have no idea how to read one. Not a vague idea, not a rough intuition. No idea. Here's what they get wrong, over and over.
No feel for scale, at either end
People can't feel the difference between one in ten thousand and one in a million. Tiny numbers all blur into "it could happen." Huge numbers blur the same way: a million and a billion sound like roughly the same pile of money. They aren't. A million seconds is under 12 days. A billion seconds is almost 32 years.
This is exactly what headlines exploit with relative risk. "Doubles your risk of cancer!" sounds terrifying. Then you find out the risk went from one in a million to two in a million. The relative change is 100%. The absolute change is basically nothing. The single most useful question you can ask about any scary number is: out of how many? Almost nobody asks it.
Base rates
Here's a classic. A disease affects 1 in 1,000 people. A test for it is 99% accurate. You test positive. What are the chances you're sick?
Most people say 99%. Most doctors get it wrong too. The real answer is about 9%. Test 100,000 people: 100 are sick, and 99 of them test positive. But of the 99,900 healthy people, 1% also test positive, which is 999 false alarms. So out of roughly 1,100 positive results, only 99 are real.
When something is rare, even a very good test mostly catches false positives. Ignoring how common something is to begin with is one of the most expensive mistakes people make, in medicine, in security screening, and in court.
Mean versus median
Most of the public can't tell a mean from a median, and politicians love it. "Average income went up" can be true while most people got poorer. Put a billionaire in a room with a hundred factory workers and the average person in that room is a multimillionaire. The median, the person in the middle, hasn't changed at all.
Whenever a distribution is lopsided, and income, wealth, house prices and hospital stays all are, the mean tells you about the outliers and the median tells you about the typical person. Whoever chooses which one to report is choosing the story.
Statistical categories are not people
Thomas Sowell makes a point that almost nobody understands. "The rich got richer and the poor got poorer" compares brackets, like the top 20% and the bottom 20%. But brackets aren't people. People move in and out of them all the time.
A lot of the bottom fifth are young workers at the start of their careers. Ten years later many of them have moved up, and new young workers have taken their place. So "the bottom 20% didn't get richer in a decade" can be true even though most of the actual people who were in it got a lot richer. The bracket stayed poor. The people didn't. Confusing a statistical category with flesh-and-blood human beings produces some of the worst economic arguments around.
Correlation, small samples and cherry-picked graphs
Two things moving together doesn't mean one causes the other. Ice cream sales and drownings both rise in summer; ice cream doesn't drown anyone. Add tiny samples ("a study of 20 people found...") and graphs whose axis starts wherever makes the line look dramatic, and you have most of what passes for evidence on social media.
Regression to the mean
Wherever there's randomness, extreme results tend to be followed by more ordinary ones. That's it. No cause needed. But people always find a cause. Speed cameras get installed at a junction right after a terrible year of crashes. The next year crashes drop, as they would have anyway, and the cameras get the credit. A team fires its coach after a disastrous season, the next season is better, and the new coach is a genius. Half of what gets called "the policy worked" is just this.
Simpson's paradox
A trend can show up in every subgroup and flip once you combine them. The famous case is Berkeley admissions in 1973. Overall, men were admitted at a higher rate than women, and it looked like obvious bias. Department by department, though, women did as well or better. They had simply applied more to the competitive departments that rejected most applicants. The aggregate number told the opposite story from the real one.
The prosecutor's fallacy
"There's only a one-in-a-million chance an innocent person would match this DNA" gets heard as "there's a one-in-a-million chance he's innocent." Those are completely different statements. In a country of 10 million people, you'd expect about 10 innocent people to match. Without other evidence, the match alone makes him one of roughly 11 suspects, not a near-certain culprit. People have gone to prison over this confusion.
Margin of error
A candidate goes from 48% to 50% in a poll and the news calls it momentum. With a typical margin of error of about 3 points, that change is noise. People treat every wiggle as a story, and the media is happy to sell it to them.
The gambler's fallacy
After five reds in a row, people bet on black because it's "due." The roulette wheel has no memory. Neither does a coin, a die, or the genetics of a family that keeps trying for a boy. Independent events don't keep score, but people insist they must.
Why this matters
This isn't trivia. Voters are asked to judge policies that are, at bottom, statistical claims: about crime, health, inequality, the economy. A public that can't tell a mean from a median, or relative risk from absolute risk, can be told almost anything with a straight face, and it is.
Meanwhile schools drill students in calculus that most of them will never use again, and barely teach the statistics they'll need every time they read a headline, see a doctor, or sit on a jury. If we taught people to ask "out of how many?", "compared to what?" and "what's the base rate?", we'd have a far harder population to fool. Right now, we have a very easy one.