What is a Z-Test?

A z-test judges whether an observed difference (a mean against a benchmark, a proportion against a claim, two proportions against each other) is larger than sampling noise would produce, by locating the observed result on the standard normal distribution: the z-statistic counts how many standard errors separate observation from the null's expectation, and the familiar machinery converts that distance into a p-value. Its defining assumption is that the sampling distribution is normal with a known spread, which holds in two practical situations: when the population variability is known in advance (rare outside quality control), and, far more commonly, when samples are large enough that the Central Limit Theorem makes the normal approximation excellent and the sample's own variability estimate essentially exact.

Z or T?

The relationship to the t-test is close and historical. Gosset built the t-distribution precisely because, at small samples, estimating the variability from the data adds uncertainty the normal curve ignores; the t's heavier tails price that in, and converge to the z as samples grow. The practical rule is undramatic: small samples of continuous measures take the t (and modern software defaults to it regardless, at no cost); large-sample work, and especially proportion comparisons, traditionally take the z. Since conversion rates, click-through rates, and completion rates are proportions measured on thousands of users, the two-proportion z-test is the classical engine behind most A/B testing verdicts, whether or not the dashboard says so, with the chi-square test as its algebraic twin on the same data.

Using It Well

1. Check the scale of n.
The z-test's licence is size: for proportions, the standard sanity check is enough expected successes and failures in each group (a handful at minimum, comfortably more in practice). Below that, exact methods serve better.

2. Keep observations independent.
The universal fine print: repeated events from the same users are not independent trials, and session-level counts often smuggle this violation into proportion tests.

3. Decide one-sided or two-sided before looking.
The same pre-commitment rule as everywhere in testing; choosing the favourable tail after seeing the data is a classic quiet cheat.

4. Report magnitude with the verdict.
A z-test on a huge sample will flag differences of no commercial consequence; the difference in proportions with its confidence interval is the decision-grade output, with all the standing cautions of statistical significance.

The Takeaway

The z-test is the large-sample original: distance from the null, measured in standard errors, priced on the normal curve. Let the t handle small continuous samples, give the z the big proportion comparisons it was made for, honour independence and pre-commitment, and read its verdicts alongside the magnitudes that actually steer decisions. Most days, it is the statistics running quietly under your experiment dashboard's green light.