What is Generalizability?

Generalizability is the extent to which research findings apply beyond the specific sample, context, and time in which they were produced: the external validity question, phrased as a property of the results. It has three dimensions. Population: from the participants to the users at large. Setting: from the study's tasks, prototype, and environment to real use. Time: from this quarter's behaviour to next year's. A finding can travel well on one axis and badly on another: a perfectly representative survey of today's customers says nothing reliable about next year's prospects, and a flawless lab task generalises poorly to a commuter on a cracked phone.

Two Traditions, Two Routes

Statistical generalisation is the quantitative route: a representative sample drawn by probability methods from a defined population licenses inference to that population, with uncertainty priced by the margin of error. Its scope is exactly as wide as the sampling frame and no wider, and every convenience sample truncates it silently.

Analytic generalisation (Lincoln and Guba called the qualitative equivalent transferability) is the other route: small, purposively chosen samples don't estimate prevalence, but they can identify mechanisms, patterns, and categories that recur wherever similar conditions hold. A usability problem found with five participants generalises not because five is a magic number but because the problem lives in the design, and the design is the same for everyone who meets it. The qualitative researcher's obligation is thick description: enough detail about participants and context that readers can judge whether their situation resembles the studied one.

The Common Failures

Sample-to-population overreach. Prevalence claims ("40% of users want X") from twelve interviews, or from a panel that was never randomly drawn; the frame decides the reach, not the confidence.

WEIRD defaults. Behavioural science's own reckoning (the 2010 critique that most psychology samples were Western, educated, industrialised, rich, and democratic, and were nonetheless treated as human universals) has a product-research twin: samples of early adopters, internal staff, or one market generalised to everyone.

Setting blindness. Prototype findings treated as production findings; moderated-lab behaviour treated as unobserved behaviour, the observation problem in generalisation form.

Temporal drift. Research ageing into folklore: the study from three product versions ago still cited as current truth.

Earning and Stating Reach

1. Match the claim type to the design.
Mechanism and problem-discovery claims from small purposive samples; prevalence and magnitude claims from representative ones; causal claims from experiments. Most generalisation errors are claim-type mismatches.

2. Sample for the reach you want.
For statistical claims, invest in the frame; for transferability, sample deliberately for range (contexts, expertise levels, markets) so the mechanism is seen under varied conditions.

3. Replicate and triangulate.
The same finding in a second sample, a different method, or another market is the strongest generalisation evidence available; convergence across failure modes beats any single study's reach.

4. Move toward real conditions.
Remote unmoderated studies on participants' own devices and contexts (the default Ballpark format) narrow the setting gap the lab opens; staged rollouts close it.

5. Write the scope sentence.
Every readout carries one: "Findings describe [who], doing [what], in [context], as of [when]; we'd expect them to transfer to [conditions] and would test before assuming [beyond]." It converts a shrug into a claim.

The Takeaway

Generalizability is reach earned by design and stated by discipline: statistical inference where the frame allows, analytic transfer where mechanisms recur, replication wherever the stakes justify it, and a scope sentence on every finding. Small studies can travel far when they claim the right things; large ones can't travel at all when they claim the wrong ones.