The risks associated with the user interface of a web page or software application are tremendous. We know, for example, that usability problems in a system will significantly decrease end user efficiency and increase support costs. End users also tend to resist adopting systems with poor usability, so the competitor with the easier user interface gets the edge in the marketplace. And, several studies have shown that incorporating usability checks into the development process is far less expensive than redesigning a system to make it more usable late in the development cycle, or even after an initial launch. Any of these factors may significantly reduce the return on your company’s investment in developing software or web applications.
While it’s important to manage risk in the business world, it’s also important to minimize costs. While it’s not difficult to design usability tests that deliver statistically valid results, the cost of conducting these tests can be prohibitive due to the large sample sizes required, and the benefits marginal. Since we recommend that you conduct usability tests early in the design process to identify areas that need improvement, it’s usually much cheaper to just fix the problems identified by a small group of testers than it would be to statistically confirm the results of the tests with a larger population of users.
Furthermore, research has demonstrated that the number of usability problems, found in a design, levels off significantly after the first six testers. The first five test participants typically discover approximately 85% of the usability problems in a task; but it might take another ten testers to find the remaining 15%. Consequently, we find that the smaller sample size is a more cost-effective choice for most development projects, even if it does not engender statistical validity.
Plus, you can use the money you saved on running that enormous study to run additional test cycles later in the development process. This not only allows you to validate the design choices made to fix the initial problems, but also ensures that no additional usability problems are introduced by the new designs.