How many participants are really enough for usability studies?

Alroobaea, Roobaea and Mayhew, Pam J. (2014) How many participants are really enough for usability studies? In: Proceedings of 2014 Science and Information Conference, SAI 2014. The Institute of Electrical and Electronics Engineers (IEEE), GBR, pp. 48-56. ISBN 9780989319317

Full text not available from this repository.

Abstract

The growth of the Internet and related technologies has enabled the development of a new breed of dynamic websites, applications and software products that are growing rapidly in use and that have had a great impact on many businesses. These technologies need to be continuously evaluated by usability evaluation methods (UEMs) to measure their efficiency and effectiveness, to assess user satisfaction, and ultimately to improve their quality. However, estimating the sample sizes for these methods has become the source of considerable debate at usability conferences. This paper aims to determine an appropriate sample size through empirical studies on the social network and educational domains by employing three types of UEM; it also examines further the impact of sample size on the findings of usability tests. Moreover, this paper quantifies the sample size required for the Domain Specific-to-context Inspection (DSI) method, which itself is developed through an adaptive framework. The results show that there is no certain number of participants for finding all usability problems; however, the rule of 16 4 users gains much validity in user testing. The magic number of five evaluators fails to find 80% of problems in heuristic evaluation, whereas three evaluators are enough to find 91% of usability problems in the DSI method.

Item Type:	Book Section
Uncontrolled Keywords:	domain specific inspection (dsi),heuristic evaluation (he),methodological framework,sample size,user testing (ut),information systems ,/dk/atira/pure/subjectarea/asjc/1700/1710
Faculty \ School:	Faculty of Science > School of Computing Sciences
UEA Research Groups:	Faculty of Science > Research Groups > Data Science and Statistics Faculty of Science > Research Groups > Smart Emerging Technologies
Related URLs:	http://www.scopus.com/inward/record.url?...
Depositing User:	Pure Connector
Date Deposited:	23 Jun 2016 23:14
Last Modified:	06 Jan 2023 11:30
URI:	https://ueaeprints.uea.ac.uk/id/eprint/59573
DOI:	10.1109/SAI.2014.6918171

Actions (login required)

View Item