Understanding the Methodology

Benchmarking is built on a systematic approach, where data collection, analysis, and interpretation form the core of the process. Here is a breakdown of the key elements involved in benchmarking methodology:

1. Identifying Key Metrics: The first step is to define what will be measured. Common metrics in user research include task success rates, time on task, error rates, and user satisfaction scores. These metrics need to align with organisational goals or industry standards.

2. Establishing a Baseline: A baseline is the starting point from which future comparisons will be made. It may come from internal historical data or be based on competitor analysis. For instance, if a company is launching a new app feature, the baseline could be the performance metrics of a competitor’s similar feature.

3. Collecting Data: Data collection methods vary depending on the context, but they typically include usability testing, surveys, A/B testing, analytics, and interviews. User experience researchers gather both quantitative and qualitative data to provide a comprehensive view of the current state.

4. Analysing Results: Once the data is collected, analysis focuses on identifying strengths and weaknesses. Are users taking longer to complete a task than those of competitors? Is user satisfaction declining over time? Analysis often leads to hypotheses about why certain issues are occurring, helping to prioritise areas for improvement.

5. Iterating and Implementing Changes: Benchmarking is not a one-off process. It requires continuous evaluation, particularly after changes are made to a product or service. This iterative approach ensures that progress is tracked and the organisation moves closer to meeting or exceeding its benchmarks.

Top 5 Use Cases for Benchmarking Research

1. Measuring Product Usability Against Competitors: One of the most common applications of benchmarking is comparing the usability of a product against that of competitors. By identifying strengths and weaknesses in the user experience, organisations can adapt quickly, capitalising on areas where they outperform rivals or addressing shortcomings.

   For example, a study by the Nielsen Norman Group found that average task success rates across websites typically range between 70-80%. Companies falling short of this figure have a clear indication that usability improvements are required to remain competitive.

2. Tracking Product or Service Improvements Over Time: After a product launch or feature update, benchmarking can be used to track progress. It is especially valuable for companies with a long-term focus on optimising user experiences. By establishing a benchmark before and after a redesign, researchers can measure the impact of their efforts in quantitative terms.

   Google, for instance, regularly uses internal benchmarks to evaluate the effectiveness of new feature rollouts. A case in point is YouTube’s autoplay feature, where Google benchmarked its impact on session lengths and engagement.

3. Optimising Customer Journeys: Understanding the customer journey in-depth is crucial for businesses seeking to create frictionless experiences. Benchmarking is invaluable for comparing various touchpoints within the customer journey, from checkout processes on e-commerce platforms to onboarding in SaaS products, or issue resolution in customer service environments.

4. Improving Accessibility: Benchmarking is also useful in assessing how well a product meets accessibility standards. By comparing performance metrics for users with disabilities against industry guidelines, businesses can identify gaps and enhance the inclusivity of their designs. At a time when digital accessibility is a legal and ethical priority, benchmarking helps organisations comply with regulations such as the UK Equality Act while fostering a more inclusive user base.

5. Benchmarking Satisfaction and Loyalty: Many companies use Net Promoter Score (NPS) and Customer Satisfaction (CSAT) scores as benchmarks to gauge user loyalty and satisfaction. Regularly benchmarking these scores provides vital insights into how product or service changes impact user sentiment. Apple and Amazon, for example, heavily rely on customer satisfaction benchmarks to continually refine their offerings.

Best Practices for Benchmarking

To ensure meaningful outcomes, it is essential to adhere to best practices when conducting benchmarking research. Below are some strategies that organisations can employ:

1. Focus on the Right Metrics: The metrics selected for benchmarking should reflect the company’s goals and be actionable. It’s tempting to gather as much data as possible, but focusing on key metrics such as task completion rates, satisfaction, or retention rates ensures that the data collected is relevant and can lead to effective changes.

2. Use Both Quantitative and Qualitative Data: Combining quantitative data (like success rates and time on task) with qualitative data (such as user feedback) offers a more comprehensive understanding of the user experience. While quantitative data provides hard numbers for comparison, qualitative insights explain why those numbers are what they are, offering richer context for decision-making.

3. Benchmark Against Competitors and Industry Standards: Selecting appropriate benchmarks for comparison is crucial. Companies can benchmark their NPS scores against industry averages or compare usability against direct competitors. Relying on sources such as the 2023 Digital Experience Benchmark by Contentsquare, which reports on metrics like time spent per page and bounce rates across industries, can provide guidance on where improvements are most needed.

4. Iterate Regularly: User behaviour evolves quickly, and what worked last year may not be effective today. Regular benchmarking allows for continuous improvement. It is important to update benchmarks over time to keep pace with changing trends, customer needs, or the emergence of new competitors.

5. Make Data-Driven Decisions: The real value of benchmarking lies in the ability to make informed, data-driven decisions. Organisations should ensure that benchmarking results are shared with key stakeholders and used to guide product development, marketing strategies, and customer experience initiatives.

Conclusion

Benchmarking in user research is a powerful tool that enables businesses to assess how their products and services compare with industry standards and competitors. By providing a structured approach to understanding performance, usability, and customer satisfaction, benchmarking informs product development and improvements that lead to better user experiences.

Adopting best practices such as focusing on the right metrics, combining quantitative and qualitative data, and conducting benchmarking on a regular basis ensures that the process is effective and insightful. Whether it’s used to track performance over time, improve accessibility, or optimise the customer journey, benchmarking remains an essential technique for businesses committed to remaining competitive in a rapidly evolving digital landscape.