What is the Margin of Error?

The margin of error expresses the uncertainty in a sample estimate due to random sampling variation: the range around the reported number within which the true population value plausibly sits, at a stated confidence level (almost always 95%). "52%, ±3 points" is a compressed confidence interval: the survey's best estimate is 52, and the procedure that produced it captures the truth within three points either way in about 95 of 100 repetitions. The margin is the half-width of that interval, and everything about its correct reading follows from that definition, including its most important property: it quantifies the wobble of random sampling, and nothing else.

What Moves It

Sample size, with diminishing returns. The margin shrinks roughly with the square root of n: around ±10 points at 100 respondents, ±3 at 1,000, ±2 at 2,500 (for a proportion near 50%, at 95% confidence). Halving the margin costs quadruple the sample, which is why precision is a budget decision, and why the jump from 100 to 400 respondents buys more than the jump from 1,000 to 1,300.

The value being estimated. Proportions near 50% are the noisiest; margins tighten somewhat as results approach 0% or 100%. Quoted margins conventionally assume the worst case.

The population size, barely. Counterintuitively, population size is nearly irrelevant once the population is large: a well-drawn sample of 1,000 carries roughly the same margin whether the population is a million or a hundred million. Soup tasting works the same from any size pot, provided the pot is stirred, and the stirring is the random sampling the whole calculation assumes.

Subgroups, brutally. The quoted margin belongs to the full sample. Slice to a segment (one plan tier, one region) and the margin balloons to that slice's own n; a ±3 survey becomes a ±10 statement about its 100-person subgroup. Most overinterpretation of survey crosstabs is this arithmetic, skipped.

What It Does Not Cover

The margin of error is silent about every systematic problem: a skewed frame (sampling bias), the self-selection of who answers (non-response bias), leading questions and dishonest answers (response bias), and plain question misreading. A survey can be biased by ten points and still truthfully report ±3, because the three covers only the sampling wobble around whatever (possibly tilted) value the method converges on. Pew Research Center's explainer on polling margins makes the point plainly: total survey error is always larger than the quoted margin, and the quoted margin is simply the one component mathematics can price.

Using It in Product Research

1. Report it, at product-research sample sizes especially.
A 60% task-success rate from 20 participants carries a margin north of ±20 points; saying so converts false precision into honest evidence. The same discipline applies to SUS scores, NPS, and every dashboard proportion.

2. Size studies backwards from the decision.
Decide the precision the decision needs, then buy that much sample, and no more. A directional read tolerates ±10; a pricing decision may not.

3. Overlap is a warning, not a verdict.
Two estimates whose margins overlap may still differ reliably (the proper comparison is a test on the difference), but heavily overlapping intervals from small samples mean the honest headline is "too close to call", however the point estimates order themselves. The machinery of statistical significance exists for exactly these judgments.

The Takeaway

The margin of error is the honest label on sampling's luck: read it as the plus-or-minus of random draw, watch it explode in subgroups, and never mistake it for a warranty against bias, which it cannot see. A number wearing its margin is a measurement; a number without one is a mood.