Statistical fallacies that cost millions (and how to spot them)
Why smart people fall for the same 8 data traps
Google thought they'd cracked the code on predicting flu outbreaks. Using billions of search queries, they found their algorithm could spot flu trends a few weeks ahead of the CDC.
Then came the 2012-2013 flu season. Google Flu Trends overestimated flu cases by 100%, predicting double the doctors' visits as the CDC actually recorded. The algorithm had erroneously identified high school varsity basketball as one of the top predictors of flu. The (winter) basketball season coincided with (winter) flu season, so the model assumed one predicted the other.
And this wasn't a small startup mistake. This was Google — with all their PhDs, computational power, and billions of data points — falling for one of the oldest statistical fallacies in the book. If it can happen to them, it can happen to your data team.
The dangerous allure of patterns
I gravitated to math from an early age, eventually studying applied math at Princeton (go ORFE!) and working as a quant. I use mathematical analogies to describe life and write math papers to improve at tennis. Numbers felt like the closest I could get to understanding reality.
Then I noticed something unsettling: even the most mathematically sophisticated people fell for certain statistical traps. Nobel laureates at Long-Term Capital Management. PhDs at Google. Brilliant quants who could derive Black-Scholes in their sleep, yet somehow missed these minefields in their own work.
If these mathematical heavyweights could stumble into such pitfalls, what chance do the rest of us have? Quite a bit, actually, but only if we know what to look for.
These aren't intelligence failures — they're human failures. We evolved to see tigers in the grass, not to interpret regression analyses. Our brains are wired to find patterns, even when they don't exist. And that wiring doesn't disappear just because you've mastered stochastic calculus.
When math goes wrong
Math is often indistinguishable from magic.
For example, all the fancy AI and machine learning stuff where we can get large language models to chat with us? Fundamentally, that's all math: high-dimensional vector spaces, matrix multiplication, and some other cleverness sprinkled in. Oh, and GPUs. Lots of GPUs.
But this same mathematical power becomes destructive when misused. Target's pregnancy prediction algorithm triggered a $23 million PR nightmare. Long-Term Capital Management's "perfect" models nearly crashed the global financial system, requiring a $3.6 billion bailout. A slight statistical mistake can cost serious money.
People make the same systematic errors over and over. These mistakes are so common they have names, ones you'll recognize once you know the signs.
The fallacies that will fool you
Over the next few weeks, I'm going to break down the most dangerous statistical traps I've encountered. Each one has cost real companies real money, derailed promising careers, and led to spectacularly wrong decisions.
Correlation is not causation: Just because two things move together doesn't mean one causes the other.
Overfitting: If your model is so perfectly fitted to past data, it will completely fail in the real world.
Data snooping: The practice of mining vast quantities of data until you find "significant" patterns.
Base rates: Why a 90% accurate cancer test might mean you only have a 15% chance of actually having cancer.
Levels v. differences: The subtle but devastating mistake of confusing levels with differences in time series data.
Axis truncation: How zooming in on data makes tiny shifts look massive — see it once and you'll see it everywhere.
Survivorship bias: Why we only hear success stories and how this creates dangerously misleading advice.
Narrative fallacy: Our brains are wired for stories, but stories are terrible models for predicting complex reality.
Why this matters more than ever
Whether you're a product manager analyzing user engagement, a marketing director reviewing campaign performance, or an executive making budget decisions, you're swimming in these statistical waters every day.
That "breakthrough" pattern your analyst just found? That correlation your consultant is pushing? There's a good chance it's one of these eight fallacies in disguise.
When storage costs pennies and computers churn through petabytes overnight, we generate patterns faster than we can vet them.
Data drives every modern decision, and complexity only climbs. The stakes have never been higher.
Your statistical survival kit
This series will give you:
Real-world examples that make the concept stick
Warning signs to help you detect the trap in practice
Actionable frameworks for better decision-making
Short ‘gut‑check’ questions you can apply in under a minute
By the end, you'll join the small group of professionals who can spot statistical nonsense before it metastasizes into million-dollar mistakes.
Master these, and you'll have something more valuable than mathematical sophistication — you'll have statistical wisdom.
This is Part 1 of our 9-part series on statistical fallacies. Subscribe below for more:
Next time: We'll start with perhaps the most famous statistical fallacy of all — the dangerous assumption that correlation implies causation. I'll show you why two perfectly correlated advertising platforms can have completely different business value, and how this mistake has torpedoed countless business strategies.
Catch up on the entire series: