Why Should I Only Have 5 Participants In My Usability Study?

You should only have 5 participants in your usability study. Any more is a waste of your time and resources!

Want to find out why? Keep reading this article to learn more.

What Is The Neilson Norman Group’s 5 Rule?

As per the esteemed Nielsen Norman Group, conducting tests with just 5 individuals can help uncover nearly as many usability issues as testing with a larger group of participants. The rationale behind their recommendation of 5 users is based on the diminishing returns of testing more people. Beyond this point, the additional insights gained come at a higher cost. After testing with the initial 5 individuals, further participants tend to highlight the same usability issues with minimal significant variation. (source)

Has This Been Proven?

While this has been statistically proven, unfortunately for us Math haters, we’re going to have to delve into some statistics to understand why.

Considering the likelihood of a user encountering an error during testing stands at 31%, as indicated by Jeff Sauro of MeasuringU, it’s worth noting that testing with just 5 users can reveal a substantial 85% of interface problems. This conclusion is drawn from the principles of binomial probability, specifically the Poisson Distribution, which helps us understand the probability of achieving n successes in N trials.

A Poisson Distribution with a 31% binomial probability demonstrates that once you exceed 5 users in a test group, the returns diminish significantly. In other words, the more users you add to your test group, the less additional insights you’ll gain.

To put it in perspective, with 3 test users, you’ll detect approximately 65% of the problems. Increasing to 4 users brings the detection rate to 75%, and with 5 users, you’re already capturing 85% of the issues.

However, as you surpass the 5-user threshold and continue adding more participants, the rate of uncovering new issues declines considerably. For example, with 6 users, you’ll capture 90% of the issues; with 8 users, it’s 95%; and with 12 users, you’ll reach 99%. These diminishing returns are primarily due to the fact that user experiences tend to overlap.

As explained by the Nielsen Norman Group, your insights significantly increase after gathering data from just one test user, accounting for nearly a third of what you need to know about the design’s usability. Subsequent users will exhibit some similarities to previous users, but also introduce some new behaviours.

While people are unique, the third user will reiterate many of the actions observed with the first two users, with only a small amount of genuinely new data. This trend continues with each additional user, with the most significant plateau occurring at 5 test users, making this number a suitable balance between sample size and value gained.

When Should I Use More or Less Than 5 Participants?

There are situations where the conventional 5-user guideline may not be suitable. This is especially true for products that demand a more extensive participant pool to yield meaningful test results. For example, complex e-commerce websites may require a larger sample size as testing just five participants might uncover only 35% of their usability issues. Moreover, your usability study’s participant count may need adjustment if you:

• Address Diverse User Segments: When your product caters to distinct user groups with varying behaviours, it’s essential to test individuals representing each group. For instance, a marketplace app may serve both buyers and sellers. In such cases, you don’t necessarily need five participants for every user category, as there may be overlapping observations. Testing with as few as 3 or 4 participants for each group could suffice.
• Seek Quantitative Data: In quantitative usability studies, statistical significance is crucial for obtaining precise insights. Typically, you’ll need a larger participant pool, ranging from 20 to 40 individuals. Since the focus is on quantifiable metrics rather than qualitative findings, a substantial number of participants is necessary to collect enough data for accurate predictions about your overall target users’ behaviours.
• Follow an Agile UX Approach: Agile UX methodologies involve multiple usability tests as the product evolves through iterative development. In this dynamic process, you conduct tests, leverage acquired insights to make improvements, and then retest the revised version, iterating this cycle. Instead of using five or more participants for each test iteration, you can run multiple tests with as few as 3 participants each. This approach capitalises on overlapping insights and discoveries, while also providing cost savings for your study.

Advantages of Using The 5 Rule for Usability Testing

Researchers have proposed varying optimal participant numbers throughout the years. Maintaining a minimum of three participants guarantees the inclusion of diversity within your user group. Conversely, having as many as seven participants ensures that nearly all usability issues within the product are discovered. Therefore, the ideal number of participants typically falls within a range of three to seven, and this range is recommended for the following reasons:

It Maximises Returns

In the context of usability testing, the law of diminishing marginal returns dictates that as you increase the number of participants in your study, there comes a point where the additional participants contribute progressively fewer new insights. This concept is effectively illustrated in a graph presented by the Nielsen Norman Group (above).

To clarify, in the example provided:

• Testing the first participant yields valuable, fresh insights.
• When testing the third participant, numerous issues may surface that the first two participants might have overlooked.
• Even when you reach the sixth participant, you continue to uncover new problems that the initial five participants may have missed.

However, as you progress to the 12th participant, there is a notable decline in the acquisition of new insights, as most usability problems have already been identified. For practical purposes, testing with a group ranging from 3 to 7 participants strikes the optimal balance. Within this range, you can obtain sufficient insights without succumbing to the diminishing marginal returns phenomenon.

Cost Efficiency

When it comes to product development, one of the primary reasons why usability testing encounters resistance from stakeholders is the financial aspect. Conducting usability tests incurs expenses that can vary significantly based on factors such as the type of test, the number of participants, the product’s complexity, and more. These costs can range from as little as $250 U.S. dollars to as much as $10,000 U.S. dollars, or even higher.

These expenses encompass various elements, including fees for recruiting agencies, software expenditures, and the provision of incentives like gift cards for participants. Running a single usability test involves not only visible costs but also hidden ones. Given that expenses tend to escalate with the number of participants involved, conducting tests with a limited number of individuals becomes an effective strategy for cost minimization while optimising your budget.

Time Efficiency

Usability tests are typically time-intensive, involving various phases, from initial setup to conducting test sessions and then analysing the outcomes. The duration of this process depends on factors such as the number of participants and whether the study is moderated or unmoderated. It can range from a single day to several days for completion.

For instance, it’s estimated that conducting usability tests with just 5 participants could demand a considerable time investment, spanning anywhere from 11 to 48 hours. Instead of dedicating substantial time and resources to test a larger number of participants for your study, you can achieve comparable results in significantly less time by focusing on testing only a select few participants.

Overview

Within the realm of usability testing, the “5-Participant Rule” emerges as a valuable guideline, offering a pragmatic and efficient strategy for uncovering significant usability issues. This rule revolves around striking a delicate balance, one that harmonises the imperative for thorough testing with the constraints of time, budget, and available resources.

By conducting tests with just five participants, you can adeptly pinpoint critical usability problems at an early stage in the development process. This method presents an economical means of detecting major issues before they escalate, leading to resource savings and the prevention of costly redesigns down the line.

Nevertheless, it’s vital to bear in mind that the 5-participant rule isn’t a one-size-fits-all solution. Its effectiveness can vary depending on factors such as the intricacy of your product, the diversity of your user base, and the specific goals of your testing. In certain scenarios, you may find it necessary to adapt the sample size or complement your findings with qualitative insights to ensure a comprehensive evaluation.

In essence, usability testing is an adaptable and dynamic process. The key lies in using the 5-participant rule as a launching point rather than a rigid dictate, remaining receptive to adjustments and refinements as circumstances dictate. Ultimately, the objective is to craft user-friendly products that resonate with your audience, and usability testing stands as a potent tool on that journey. So, embrace this guideline, iterate, and fine-tune your usability testing approach to shape outstanding user experiences that distinguish your products in the market.

Now that you know the ultimate number of people to include in your usability study, check out our guide to the Mom test, where you only need a single individual. Alternatively, see our guide to closing the customer feedback loop which will help you gain feedback once you’ve already launched your product.