Glossary

Random Sampling

Glossary

Random Sampling

Introduction

The most counterintuitive idea in research methodology is that the fairest way to choose participants is to stop choosing. Random sampling hands selection to chance, giving every member of a population a known probability of inclusion, and in exchange receives the two things judgment can never guarantee: freedom from selection bias, and the mathematical right to quantify uncertainty. This article explains why randomness works, the main designs built on it, and what to do when true randomness is out of reach, which in product research is most of the time.

What is Random Sampling?

Random sampling selects study participants by chance, such that every member of the defined population has a known, non-zero probability of being chosen. In its purest form, simple random sampling, every individual and every combination is equally likely, like drawing names from a complete, well-shaken hat. The power is in what randomness removes: no human chooser means no human favouritism, conscious or otherwise, so the systematic tilts of sampling bias have no mechanism to enter. What remains is random error alone, the wobble of chance, which, unlike bias, shrinks predictably as samples grow and can be quantified: confidence intervals, margins of error, and significance tests all borrow their legitimacy from the randomness of the draw.

The Probability Family

Simple random sampling draws directly from the full list. Systematic sampling takes every k-th name from a randomly-started ordered list, operationally easier and equivalent in practice unless the list has a hidden rhythm. Stratified sampling divides the population into groups (strata) first (by plan tier, region, usage band) and samples randomly within each, guaranteeing representation of every stratum and typically improving precision; it is the workhorse when subgroup comparisons matter. Cluster sampling randomly selects whole groups (schools, branches, accounts) and studies within them, trading some precision for enormous practicality when no individual-level list exists. All four are probability methods; all four support honest inference. Their common prerequisite is the sampling frame: an actual list of the population, whose gaps become the study's gaps regardless of how beautifully you randomise from it.

Randomness in Product Research

True random sampling appears in product work more often than teams notice. Randomly sampling your user base for a survey invitation is the honest version of "we emailed some customers"; it converts an anecdote collection into an estimate. The other great application is allocation: an A/B test randomly assigns users to variants, which is random sampling's logic applied to treatment rather than selection, and the entire basis of its causal authority. Where the method strains is small-n qualitative work: randomly drawing five interviewees from a database sounds rigorous but delivers neither statistical power nor guaranteed relevance; qualitative studies are usually better served recruiting purposively for the behaviours the question needs, via screeners, and being honest that the claim is insight, not incidence.

Practising It Honestly

1. Build the best frame you can, and name its holes.
Random selection from an incomplete list is random among the listed. Know who your frame misses (churned users, free tier, non-customers) and scope the findings accordingly.

2. Randomise mechanically.
Use a random function, not human judgment "spread around fairly"; humans are dreadful randomisers and excellent unconscious selectors.

3. Stratify when subgroups matter.
If the readout will compare plans, regions, or personas, guarantee each stratum its sample rather than hoping chance provides, and enjoy the precision bonus.

4. Remember selection is only half the battle.
A perfectly random invitation list still yields a self-selected response list; non-response bias operates after randomisation and can undo it. Track response rates by stratum, follow up, and weight where needed.

The Benefits

Random sampling eliminates selection bias by construction, uniquely licenses the statistical machinery of generalisation, makes error a calculable quantity rather than a hope, and carries an integrity that survives hostile audiences: "randomly selected" is the one sampling claim nobody argues with.

The Limitations

It requires a frame that often doesn't exist (there is no list of "the market"), costs more than convenience, and guarantees only the absence of selection bias, not the presence of any particular subgroup in small draws (stratification exists for that). It governs who is invited, not who answers, and it is frequently invoked loosely; "randomly asked some users" usually means haphazardly, which shares none of randomness's guarantees. For small qualitative samples it is the wrong tool entirely.

The Takeaway

Random sampling is the discipline of letting chance choose so bias cannot: build an honest frame, draw mechanically, stratify for the comparisons you need, and stay vigilant about who declines the invitation. Where true randomness is unreachable, borrow its spirit (defined populations, mechanical rules, documented gaps) and scope your claims to match. The hat only works if every name is in it and the drawing hand is blind.

Further reading

For the logic and the designs:

Articles:

1. Methods 101: Random Sampling - Pew Research Center
The clearest short explanation of why random selection is the foundation of survey credibility, with the intuitions behind the mathematics.

2. Simple Random Sampling - Scribbr
The base design in detail: requirements, procedure, and when its cousins (systematic, stratified, cluster) serve better.

3. Stratified Sampling - Scribbr
The most useful refinement for product research: guaranteeing subgroup representation while keeping probability's guarantees.