People get confused about Customer Experience (CX) and User Experience (UX). While they both have critical effects on customer satisfaction and loyalty, one significant difference is that CX targets the people who pay for the product, whereas UX focuses on the end-user of the product or service.
CX is the holistic impression a brand leaves with its customers, which affects how they think about the brand – from browsing the website to talking to customer service and receiving the product/service they bought. In contrast, UX emphasises on the process when a user interacts with a product or service as part of it. A UX designer would ask questions like: “Do your users have to remember everything when using the product?”, or “Are there any words or phrases that make your users feel confused?”
Due to the significance of CX and UX, organisations are now paying more attention to the usability of their systems and the overall user experience. Usability testing, which is conducted with real users, aims to evaluate a product’s or service’s usability by identifying specific issues when users interact with the design. It furthermore provides recommendations for corresponding solutions to improve the user experience.
However, as there are various usability testing methods available to researchers, it can be difficult to find out which methods work well together to provide good insight into the usability of a system. In this article, I will focus on two methods; System Usability Scale (SUS) and post-task walkthrough – and explain why they complement each other.
Relevance of Usability Testing
Usability testing is a method that can work in the early stage of a project. Companies do not have to wait before releasing the product or publishing the design but can still get views on how real users react. This means less time and effort are needed for developers to change the code back and forth.
Secondly, by eliminating identified issues in designs, users will experience less frustration or confusion – potentially increasing the chances of turning a hesitant customer into a buyer.
Usability testing is typically carried out in the context of actual user reactions and feedback. This helps designers and developers to match their decisions to real-world use by uncovering opportunities and fixing errors.
The usability of a website is at the heart of a customer-centric approach. For example, a website that is easy for users to navigate provides an improved user experience. Improving user experience enhances customer satisfaction, which leads to repeat visits and longer website stays. Users are less likely to return to a site after a bad experience.
A usability test is a scientific and measurable way to quantify the user experience. As UX designers/researchers, our main objective is to fully understand users and their needs in order to be able to create excellent customer experience that helps to build long-term relationships.
Conducting usability testing by recruiting participants and asking them to perform specific tasks on a website allows us to gain insight into users’ feelings and thinking – and hence their overall experience. But where do you begin your study? I would like to suggest a combination of the System Usability Scale (SUS) and post-task walkthrough as preferred methods of usability testing.
What is SUS?
The SUS is a questionnaire-based tool created by John Brooke in 1986. SUS has ten predefined statements as shown in the table below. Participants rate each of the statements based on the extent of their agreement or disagreement.
System Usability Scale
This allows us to calculate an SUS score, measured out of 100 by using the formula below or through an online SUS calculator:
- For every odd-numbered question, subtract one from the score (X-1)
- For every even-numbered question, remove the score from 5 (5-X)
- Sum the scores from even and odd-numbered questions. Then multiply the total with 2.5
Based on the above formula, if all answers are neutral, the SUS would be 50. According to John Brooke, the average score for a user interface is 68, so anything above 68 essentially means that the user interface has only minimal defects and is usable. The raw results and calculations from a real-life example SUS test is presented below:
SUS Result Form
SUS doesn’t diagnose usability problems. It rather indicates if further usability testing is required, which is the case in the example above as the average SUS is below 68. A good starting point could be to go back to the question that received a below average response from most respondents. For example, questions 2, 6 and 8 have contributed negatively to the average score. Further investigation might reveal why respondents answered those questions the way they did.
Advantages for SUS
- Evident: The SUS score can explicitly show us how our participants feel about the system and whether the product is usable.
- Quick: Participants can complete the rating to SUS in less than 2 minutes.
- Free of charge: SUS is free to use for any person or organisation.
Considerations for SUS
While there are many benefits of using SUS, some of the considerations for applying this method could be:
- Although the SUS ranges from 0 to 100, they are not a percentage, which could lead to confusion.
- Some participants may misread the SUS statements as the odd-numbered questions investigate the system’s positive aspects. The even-numbered questions explore the negative aspects of the system.
- A SUS score cannot point out the exact part of the system that needs fixing.
What is Post-Task Walkthrough?
In a post-task walkthrough, the participants have to perform tasks in a recorded session and are then later asked by a moderator to reflect on their actions. After each task, the moderator plays the video recording to collect the participants’ impressions and thoughts on the system’s usability and to gather any issues they might have experienced during the test.
A post-task walkthrough is a perfect way to obtain a subjective point of view on the participant’s behaviour. By showing the video recording immediately after the test, issues can be discussed while the memory of the participants is still fresh.
The cognitive questions, asked by the moderator after playing the recording, revolve around three major points:
- What is the impact of usability on users?
- What cognitive actions are required to perform the task completely?
- What are the learning obstacles that may occur?
Advantages of Post-Task Walkthrough
There are three advantages that I think make this method stand out that I refer to as the 3 Rs: Reliable, Reusable and Reflective.
- Reliable: The questions in the post-task walkthrough are as per Jakob Nielsen’s heuristics, which are based on years of experience in usability testing. Evaluators can comprehensively identify the issues in the user interface and save time in considering an appropriate evaluation method for a usability test.
- Reusable: Once the questions are prepared, they can also be reused by any usability testing for running a post-task walkthrough.
- Reflective: This method allows evaluators to have more time to explore the reasons behind users’ performance and the participants’ understanding of the design and their overall impression.
Considerations for Post-Task Walkthrough
A limitation with this method is that it needs UX professionals in the role of the moderator to lead the session and correctly interpret user behaviour. Using online screen recording and transcription tools, which capture the participant’s responses and body language, will help the moderator overcome this challenge and will also help to save time and effort.
Conclusion
Both SUS and post-task walkthrough are reliable, free to use and easy to conduct. In combination, SUS focuses more on providing quantitative data and post-task walkthrough pays more attention to cognitive psychology and produces qualitative data.
Post-task walkthrough complements SUS as it provides possible answers to justify the results of SUS analysis. While SUS can measure usability performance in areas like effectiveness and efficiency, it will not diagnose usability problems. This is where post-task walkthrough can be utilised as it helps identify issues related to usability.
Article is originally published at https://faircg.com/code-stories/using-system-usability-scale-and-post-task-walkthrough-to-test-usability-and-functionality-of-systems/