A dashboard reports that people spend longer in the product after a redesign. The team calls this better engagement, but the extra time could also reflect confusion. Validity concerns whether the evidence supports the interpretation being made: what a measure means, whether a causal claim is warranted and how far the conclusion applies.
It is useful to treat validity as an argument supported by evidence, rather than a permanent label attached to a study or questionnaire. The same measure can support one interpretation and fail to support another, depending on the population, purpose and conditions.
Ask whether the measure represents the concept
Construct validity concerns the relationship between an intended concept and its measurement. If the concept is trust, a question about visual attractiveness is not an adequate substitute merely because attractive designs sometimes feel trustworthy. Define what the concept means in the study and consider what else could produce the score.
Evidence may come from the content of an instrument, the way participants interpret it and its relationships with other measures. A previously studied questionnaire can be useful, but changes in wording, language or context may affect whether the earlier evidence applies.
Face validity concerns whether a measure appears suitable on inspection. That can be helpful during development, but appearance alone does not establish that the score supports the intended interpretation.
Match causal and broader claims to the design
Internal validity concerns whether a causal conclusion is supported rather than explained by other factors. Random assignment can strengthen a comparison, but implementation failures, missing outcomes or interference between participants can still threaten the inference.
External validity concerns application beyond the study. A realistic task, relevant participants and an appropriate environment can help, but a study does not become universally applicable because it took place in a home rather than a laboratory. Nielsen Norman Group’s account of internal and external validity explains why both the comparison and its intended reach deserve attention.
Ecological validity is often used for the relationship between research conditions and ordinary activity. Realism can matter, yet more realism is not automatically better for every question. Nor is there a fixed rule that stronger internal validity must always weaken external validity; careful designs can support both within a defined scope.
Separate consistency from the right interpretation
Reliability concerns consistency of measurement under specified conditions. A consistently recorded click can still be an inadequate measure of understanding. Conversely, measurement error can limit the strength of an interpretation even when the intended concept is well chosen.
For the engagement example, examine what people did during the additional time, whether they achieved their goals and how the event data were collected. Interviews, task observations and outcome measures may each illuminate a different part of the interpretation. Agreement across sources helps only if the sources actually address the question and their limitations are understood.
State the claim first, then assemble the evidence it requires. That approach makes triangulation and generalizability concrete: the team can see which interpretation is supported, which remains uncertain and what a further study would need to resolve.