Usability testing evaluates the users' ability to learn and operate the geospatial design, limit user error, appeal to user aesthetics, and adhere to accessibility requirements. The Ansyah, 2023 article outlines several usability testing methods, which I will briefly describe below. However, you will also be required to read the article to develop a more detailed background on how to implement the different usability testing methods.
Evaluation Criteria
When considering the criteria to include in a usability study, you may want to consider the different elements of a geospatial design (see Schulz, 2021 from module 2):
- Basemap Quality
- Cartography
- UX/UI
- Mobile Design Conventions
- Usability
- Location Based Services
- User Tasks
- Functionality
- Navigability
- Accessibility
GOMS (Goals, Operators, Methods, and Selection rules)
A population approach for evaluating human-computer interaction (HCI), and evaluating the ability of a user to complete a task is the GOMS usability testing method. This is very similar to a cognitive walkthrough, but with the finalized version of a design instead of the prototype. The results of the GOMS testing will provide information on the completion rate of each task and/or the time to completion for each task, therefore providing valuable information on the navigability and usability of the design.
The example below is extracted from Ansyah, 2023 and shows a GOMS usability testing sheet, with the “Goals” and “methods”, which describes the tasks the users will complete as well as the steps the user needs to take to complete that task.
The GOMS usability testing approach.
| No | Goals | Action | Methods |
|---|---|---|---|
| Task 1. Find a cafe | |||
| 1.1 | Search for a location | Tap | Tap search column |
| 1.2 | Search for a location | Typing | Type keyword |
| 1.3 | Search for a location | Tap | Tap search button |
| 1.4 | Read search results | Scroll | Scroll screen to view search results |
| 1.5 | Confirm the specified location | Tap | Tap the specified location |
| Task 2. Share a location of Gubeng Station with a friend | |||
| 2.1 | Confirm the specified location | Tap | Tap the specified location |
| 2.2 | Share to the specified app | Tap | Tap share button |
| 2.3 | Share to the specified app | Swipe | Swipe to find the app to share the location |
| 2.4 | Confirm the app to share the location | Tap | Tap the app button |
| Task 3. Add Stop to the current route | |||
| 3.1 | Find the Add Stop menu | Tap | Tap additional options menu button |
| 3.2 | Find the Add Stop menu | Tap | Tap menu add stop |
| 3.3 | Find the Add Stop menu | Tap | Tap the searched location in the search results |
| 3.4 | Search the specified location | Tap | Tap search column |
| 3.5 | Search the specified location | Typing | Type keyword |
| 3.6 | Search the specified location | Tap | Tap search button |
System Usability Study
This type of usability testing focuses on “effectiveness and efficiency” and relies on a Likert scale (answers ranging from 1 to 5, with 1 indicating strongly disagree and 5 indicating strongly agree). The total sum of scores from the Likert scale answers can indicate if users thought the geospatial design was effective and efficient.
You can design a system usability test that includes 10 more questions regarding the specific details of your design including the navigability, aesthetics, accessibility, user error, and/or additional elements that would help with understanding your users' evaluation.
| Requirement | Strongly Disagree (1) | Disagree (2) | Neutral (3) | Agree (4) | Strongly Agree (5) |
|---|---|---|---|---|---|
| I like the colors used for the geospatial design | 1 | 2 | 3 | 4 | 5 |
| I found it difficult how to go back to the home page | 1 | 2 | 3 | 4 | 5 |
| I found it difficult to log in | 1 | 2 | 3 | 4 | 5 |
| The map elements are necessary | 1 | 2 | 3 | 4 | 5 |
A/B Testing
A/B testing is a “split testing” method that is useful in comparing two different systems, and requires deploying two different versions of the system to your users and gathering feedback about the usability of both systems, to determine which is most efficient.
One method for A/B usability testing is to provide two images (one for each of the versions) and asking the users specific questions about the aesthetic appeal, navigability, accessibility, and operability of both versions.
A/B usability testing can also be on a likert scale from 1 to 5, with 1 as strongly disagree and 5 as strongly agree