Introduction
Correlation versus causation is the distinction between two things that move together (users who enable notifications retain better) and one thing that actually causes the other (notifications make users stay), a distinction that dashboards, cohort tables, and driver analyses can never settle on their own. It is the single most expensive reasoning error in product analytics, because it turns a pattern into a mandate. This article maps the ways correlation misleads product teams, the four rival explanations behind every "users who do X retain better", and the experiments and habits that earn a genuine causal claim.
The Distinction, in Product Terms
Correlation means two measures move together: accounts with more integrations churn less, visitors from organic search convert more, users who complete the tutorial retain better. Causation means one measure produces the other: add an integration to an account and its churn falls. The first is a fact about your data; the second is a claim about what would happen if you intervened, and it never follows from the first alone. Every correlation on a product dashboard has at least four possible explanations: X causes Y; Y causes X (committed users are the ones who bother with integrations); a third factor causes both (larger teams both integrate more and churn less); or the pattern is noise that significance testing exists to price. Deciding among them is the whole job, and "users who do X retain better, so make everyone do X" is the sentence that skips it.
The Classic Failure Modes
Confounding. The summer heat drives both ice cream sales and swimming (hence drownings): a lurking variable manufactures the association. Product analytics is a confounder farm, because users are not randomly assigned to behaviors; the users who adopt a feature differ from those who don't in motivation, expertise, and need, and those differences (not the feature) may drive the retention gap.
Reverse causation. "Users who complete onboarding retain better" reads as onboarding causing retention; it reads equally well as already-committed users being the ones who bother to finish onboarding. The arrow's direction is an assumption until a design pins it.
Selection effects. Correlations computed on filtered populations (survivors, completers, respondents) can invert or invent relationships that don't exist in the whole, the analytic cousin of sampling bias.
Coincidence at scale. Comb enough metric pairs and spectacular correlations appear by chance; entire websites collect them for laughs. Dashboards comb thousands of pairs daily.
Earning Causal Claims
1. Randomize where you can.
Random assignment is the confounder killer: when chance decides who gets the treatment, treated and untreated groups match on everything else in expectation, and a difference in outcomes has one live explanation. This is the entire authority of the A/B test and the controlled experiment: not fancier measurement, but assignment by coin-flip.
2. Approximate experiments where you can't.
When randomization is impossible (pricing for existing customers, features already shipped), quasi-experimental methods (before/after with comparison groups, staged rollouts, natural experiments) recover partial causal leverage, with honesty about what they can't rule out.
3. Interrogate observational correlations before acting.
Ask the four-explanations question explicitly. Check temporal order (causes precede effects). Hunt plausible confounders and control for the measurable ones, knowing statistical control only reaches what you measured. Look for dose-response patterns and mechanism: epidemiology's Bradford Hill considerations, ported to product data.
4. Use qualitative work to find the mechanism.
Correlations propose; mechanisms convince. Interviews and observation reveal why a behavior might drive an outcome, turning a pattern into a testable causal story, which a subsequent experiment can settle: the full mixed-methods loop.
The Practical Discipline
Language is the first control: "associated with", "predicts", and "correlates" are earned by observation; "drives", "causes", and "increases" are earned by design. Readouts that respect the vocabulary keep organizations honest, because the sentence "power users who enable notifications churn less" and the sentence "notifications reduce churn" recommend entirely different investments, and only one of them is supported by a correlation. When the causal question matters enough to fund action, it matters enough to fund the experiment.
The Short Version
Correlation is a clue; causation is a conclusion, and the road between them runs through design: randomize where possible, approximate where not, interrogate confounders and direction everywhere, and let mechanisms from qualitative work guide which experiments to run. The ice cream never caused the drownings. The question for every correlated metric pair on your dashboard is: what's the summer?
Further reading
For the reasoning and the remedies:
Articles:
1. Correlation vs. Causation - Scribbr
The distinction with examples of confounding, directionality problems, and the designs that support causal claims.
2. A Refresher on Statistical Significance - Harvard Business Review
The companion discipline: pricing coincidence before arguing about causes.