A survey can have thousands of answers and still leave a consequential group unheard. The issue is not simply how many people declined; it is whether their absence changes the result. Non-response bias occurs when respondents and non-respondents differ in ways relevant to the estimate being made.
Unit non-response means a selected person does not participate. Item non-response means a participant leaves a particular question unanswered. Both deserve attention, but neither automatically establishes bias. A response rate describes participation; it does not, on its own, reveal the opinions or circumstances of the people who are missing.
Ask what could connect participation to the result
Imagine a fictional training platform surveying everyone who started a course. People who completed it may be more willing to discuss the experience than those who stopped after the first lesson. If the team reports satisfaction using only the replies, it may describe a different group from the one named in its conclusion.
The direction is not always predictable. Dissatisfied people may be especially motivated to respond in another setting. Avoid assuming that silence means either contentment or disappointment; investigate the mechanism that could make participation relevant to the question.
AAPOR’s standard definitions explain why response rates are useful but insufficient for assessing non-response error. A high rate does not settle every quality question, and a low rate does not quantify the size of any bias.
Use information available beyond the replies
Where appropriate data exist for the invited group, compare participation by relevant characteristics such as course progress, tenure or plan. Differences can identify a concern and guide follow-up. Matching on those characteristics, however, does not prove that respondents and non-respondents share the same unobserved attitudes.
A shorter follow-up or another suitable contact method may reach some people who did not initially answer. Treat that additional evidence carefully: those who respond to a second attempt are still not necessarily equivalent to everyone who remains absent. Late responders can be informative without serving as a universal proxy.
Review skipped questions separately. Missing answers may reflect sensitivity, confusing wording, irrelevance or the absence of a suitable option. Each explanation suggests a different improvement to questionnaire design.
Reduce the burden and report the remaining uncertainty
Use a proportionate survey length, clear invitations, accessible presentation and fair compensation where appropriate. Respect contact preferences and avoid repeated pressure. Improving participation should not come at the expense of a person’s ability to decline.
Weighting can adjust for known differences under stated assumptions, but it cannot guarantee recovery of what missing people would have said. Report the recruitment process, participation measure, exclusions and any adjustments alongside the result.
Keep this distinct from sampling bias, which can arise through coverage or selection, and response bias, which concerns how answers are produced. In the training example, a survey sent only to course completers has a coverage problem before anyone decides whether to reply. Naming the stage correctly helps the team improve the right part of the study.