Thinking about thinking aloud : an investigation of think-aloud methods in usability testing

Alhadreti, Obead (2016) Thinking about thinking aloud : an investigation of think-aloud methods in usability testing. Doctoral thesis, University of East Anglia.

Preview

PDF Download (5MB) | Preview

Abstract

In website design and engineering, the term “usability” describes how easy a website or interface is to use. As the Internet continues to grow exponentially, with millions of websites vying for users’ attention, usability has become a critical factor determining whether a website will survive or fail. If websites are not sufficiently usable, users will simply abandon them in favour of alternatives that better cater to their needs. It is therefore crucial that designers employ effective evaluation methods in order to assess usability and improve user interface design.
One of the most widely used methods of evaluating the usability of websites is the Thinking Aloud protocol, wherein users are encouraged to verbalise their experiences, thoughts, actions, and feelings whilst interacting with the design. This provides direct insight into the cognitive processes employed by users—knowledge which can then inform strategies to improve usability. However, despite the common usage of Thinking Aloud protocol in the field, the specific think-aloud procedures employed vary widely among usability professionals.
The aim of this thesis is to investigate the utility and validity of the different variations of think-aloud usability testing methods. To this end, three empirical studies were conducted, using library websites, to compare the practical benefits of the various methods. The studies measured five points of comparison: overall task performance, the experiences of the test participants, the quantity and quality of usability problems discovered, the costs of employing the method in question, and the relationship between sample size and the number of problems detected.
Study One examined three classic think-aloud methods: concurrent think-aloud, retrospective think-aloud, and a hybrid method. The results revealed that the concurrent method outperformed both the retrospective method and the hybrid method in facilitating successful usability testing. It detected higher numbers of usability problems than the retrospective method, and produced output comparable to that of the hybrid method. The method received average to positive ratings from its users, and no reactivity (a potential issue wherein the act of verbalising the cognitive process alters that process) was observed. In addition, this method required much less time on the evaluator’s part than did the other two methods, which involved double the testing and analysis time. Lastly, in terms of the relationship between the sample size and the number of problems discovered, the concurrent and the hybrid methods showed similar patterns, and both outperformed the retrospective method in this regard.
Study Two compared the performance of the classic concurrent think-aloud method with two variations on this method in which the evaluator plays a more active role—namely, the active intervention method and the speech-communication method. The results showed that these three methods enabled the identification of a similar number of usability problems and types, and showed similar patterns with regard to the relationship between the sample size and the number of problems discovered. However, the active intervention method was found to cause some reactivity, modifying participants’ interactions with the interface, and negatively affecting their feelings towards the evaluator. The active intervention method also required much greater investment than did the other two methods, both in terms of evaluators' time, and, it was estimated, in financial terms.
Study Three compared the classic concurrent think-aloud method with the co-participation method, wherein a pair of participants work together to perform their tasks, and verbalise their processes as they interact with the interface and with one another. This study found no difference between the methods in terms of task performance. However, the co-participation method was evaluated more positively by users in comparison with the classic method. It led to the detection of more minor usability problems, and performed better in terms of the relationship between the sample size and the number of problems detected. The co-participation method was, however, found to require a greater investment of time on the part of the evaluator

Item Type:	Thesis (Doctoral)
Faculty \ School:	Faculty of Science > School of Computing Sciences
Depositing User:	Jackie Webb
Date Deposited:	24 Nov 2016 11:02
Last Modified:	24 Nov 2016 11:02
URI:	https://ueaeprints.uea.ac.uk/id/eprint/61487
DOI:

Downloads

Actions (login required)

View Item