However, doing it the right way to generate high-quality insights is not as easy as often advertised, and many teams fall into traps that aren’t talked about much. Drawing from years of experience partnering with dozens of UX research teams, I’ve identified five critical mistakes that have derailed many a prototype study. By avoiding these, you can ensure your testing delivers the insights you need and actually drives the product improvements you’re working so hard for. 

Before we go into the specifics, it’s important to understand what is actually meant by prototype testing in UX research. A prototype serves as a design artefact, similar to a hypothesis - it’s an option that may or may not solve a user problem. Prototypes can vary greatly in terms of their medium (paper, digital, or even 3D for AR/VR experiences), and specifications like level of detail and complexity. For the purpose of this article, we’ll focus on digital prototypes created with popular design tools like Figma.

Mistake #1: Using a one-size-fits-all approach to prototype fidelity

One of the most common mistakes I see in prototype studies is that most teams stick to the same fidelity level for every study and research question. This approach is problematic for two main reasons: it’s often highly unsustainable (imagine high-fidelity prototypes even in the earliest stages where details don’t matter yet) and it can cause you to miss crucial insights by not adapting to the specific stakeholder needs at the time (for example by leaving out parts of a user flow during late-stage testing because they seemed “all good” in earlier tests).

Instead, a best practice is to tailor your prototype fidelity to each research questions you’re trying to answer:

• Low-fidelity prototypes are ideal for demonstrating core functionality, information architecture, and user flows. Use these in early stages to explore concepts and primary user goals.

• Medium-fidelity prototypes add more realistic layouts, expanded user flows, and assets that much closer resemble the end product. These are great for refining and improving your product or service.

• High-fidelity prototypes are very functional and interactive, coming very close to the final product. Use these in late stages of the product development process to finalise complete flows and assess whether the feature or process as a whole is ready to be shipped to users.

Keep in mind that ideally, prototype testing should not be a one-off activity but an iterative process, progressing from low to high fidelity as your product evolves. That way, you’ll be able to address different aspects of the user experience in different intensities at each stage, making sure you get the full picture.

Mistake #2: Not setting participant expectations properly

Most UX researchers can relate to the feeling of frustration when participants don’t understand that they are actually using a prototype, especially when you did add a disclaimer before. This can be complaints about non-clickable elements, sharing confusion about paths that don’t lead anywhere, criticising “funny” placeholders, etc. Before throwing your hands in the air and condemning the method, consider two key factors at play:

1. Compared to UX professionals, participants are often completely unfamiliar with prototypes, so expressing surprise or frustration is something we can never completely avoid.

2. Humans have limited attention spans so we all easily forget instructions, especially when presented with a lot of information at once.

To tackle these challenges, consider adding disclaimers at multiple points across your study, for example at the beginning and just before each task starts. Doing so, place the disclaimers on separate pages from task info so they get undivided attention and are not just skipped over. And if you need to be extra sure, consider asking participants to confirm their understanding of the nature of the prototype before the study starts.

Mistake #3: Providing too much or too little guidance

Finding the right balance between guiding participants and allowing them to explore naturally is a very tricky challenge: Over-directing participants can bias their focus, while providing no direction may result in a lot of irrelevant data due to them looking at and commenting on aspects you don’t actually need insights on. The key lies in crafting powerful scenarios and thoughtful task descriptions. 

One of the best ways to do this is by investing some time into storytelling to create realistic contexts for your tasks. For example, instead of simply asking participants to look at a screen with options to book a medical appointment and share their thoughts, provide a scenario like this: “Imagine you’re feeling ill and need to book a check-up appointment with a General Practitioner as soon as possible. You’re having a weak signal on your phone, so you want to find options for booking the appointment online instead of making a phone call.”

Additionally, always think about and control where participants should start in your prototype. Many studies generically start on the homepage when the actual research question focuses on a feature or flow that’s much further down a funnel.

This can lead to huge amounts of data with little impact and a poor participant experience when tasks take a lot of time to complete - with the added “bonus” that once participants get to the part you’re most interested in, they might already be exhausted or have lost interest in the study.

Mistake #4: Running studies with prototypes that aren’t ready

Looming deadlines and the need to reduce time to insight are a universal pain point of UX researchers, however, they should not lead to launching a study with a prototype that isn’t ready. Although this might not be easy to hear, the hope that doing so will still yield high-quality insights is not your friend here.

Using a prototype to answer a question it can’t answer (e.g. because areas aren’t linked or there are placeholders instead of actual content) can and will compromise the results in obvious ways (e.g. participants being unable to select options essential to the experience) and less obvious ways (e.g. biased results from unrealistically easy tasks).

Generally, it’s best to postpone a study until the prototype is truly ready so you as the researcher or designer can make sure it can deliver on the study goals and scope. 

Mistake #5: Missing the details

Even with a well-designed prototype and carefully crafted tasks, there are aspects that might seem like small details but have the power to heavily compromise your results. To ensure your prototype study doesn’t leave you with a “meh” feeling after it has finished, pay attention to these aspects: 

• Optimise prototype loading times to prevent study abandonment and skewed results due to participant fatigue. This can be done through creating a “clean” testing version of the prototype without the history of iterations and other elements irrelevant to the study.

• Ensure your intended task flow is accurately reflected in the prototype, leaving out duplicates or unnecessary design elements.

• Use clear, jargon-free language and avoid placeholders like “Lorem ipsum” that might confuse participants.

• Disable all elements that could be leading such as hotspots that give away clickable areas, page titles like “Upsell Landing Page”, comments from colleagues, and your prototyping tool’s UI to avoid distractions.

By addressing these, you’ll create a seamless experience for participants, reduce the risk of biased results, and overall gather more reliable data.

In conclusion, great prototype studies require more than just creating a design and asking for some feedback. By avoiding these five mistakes - using a one-size-fits-all approach to fidelity, not setting participant expectations properly, providing too much or too little guidance, using prototypes that aren’t ready, and overlooking crucial details - you can elevate the quality and usefulness of your prototype testing efforts.

Remember, if the goal is to gather actionable insights that drive meaningful improvements in your product development, you need to make sure you’re planning and executing this method carefully so you’re truly getting the most out of it.