What is Internal Validity?

Internal validity is the degree to which a study supports the claim that its independent variable caused the change in its dependent variable, rather than some rival explanation. It is the "did X really cause Y?" chapter of validity, distinct from external validity (does it hold elsewhere?), and the two trade against each other: control buys internal validity and strips away realism. Randomised experiments purchase it by construction; every other design earns it by argument, threat by threat.

The Classic Threats

Donald Campbell and Julian Stanley's 1963 catalogue remains the checklist. History: an outside event coincides with the treatment (a marketing push lands mid-test). Maturation: participants change on their own (users get better with practice regardless of the redesign). Testing: measuring people changes them (the pre-survey primes the post-survey). Instrumentation: the measure itself shifts (the analytics definition changed in week two). Regression to the mean: extreme groups drift toward average on their own, so the worst-performing cohort "improves" after any intervention. Selection: groups differed before treatment (the users who opted into the beta were already the keen ones). Attrition: differential dropout re-selects the groups mid-study. Diffusion and contamination: the treatment leaks across arms (one team, two variants, shared screenshots). Each is a story in which something other than X explains Y, and a design's internal validity is measured by how many of these stories it has closed off.

The Design Moves

1. Randomise assignment.
Random allocation neutralises selection and balances the unmeasured, which is why the A/B test holds the causal high ground; the remaining threats are implementation problems, not design ones.

2. Use a concurrent control.
A comparison group measured at the same time absorbs history and maturation: whatever happened to the world happened to both arms. Before/after designs without a control are exposed to every temporal threat at once.

3. Hold the instrument constant.
Same measures, same definitions, same scoring rubric across arms and periods; log any change to the metric pipeline, because instrumentation drift is the most common silent threat in analytics-based studies.

4. Guard the arms.
Check attrition by arm, prevent contamination (assign at the unit that actually experiences the treatment, such as a team or account rather than a user within it), and verify the treatment was delivered as designed.

5. Blind where judgment enters.
Blinding participants and analysts to condition closes the expectancy threats the classic list underweights.

6. Where randomisation is impossible, argue explicitly.
Quasi-experimental designs answer the threats one at a time: pre-trends for selection, comparison series for history, logged co-interventions for confounding, with the residual uncertainty stated.

Internal Validity in Product Research

The threats have product-shaped disguises worth naming: novelty effects (maturation in reverse: variants win because they're new, then fade), release-week confounds (history), redefined dashboards (instrumentation), beta self-selection (selection), and cohort comparisons across time (every temporal threat together). Comparative usability studies face their own versions, handled by randomising prototype order, standardising tasks, and blinding participants to which design is "ours". The cheap, universal protection is a written design statement that walks the threat list before fielding; most internal-validity failures are threats nobody checked, not threats nobody could have closed.

The Takeaway

Internal validity is causal credibility: the observed effect belongs to the studied factor because the rivals were designed out. Randomise, control concurrently, hold instruments steady, guard the arms, blind the judges, and argue the residual threats explicitly where randomisation can't reach. Campbell and Stanley's list is sixty years old; the experiments it still catches were run last week.