- User flows and/or Journey Maps
- Video conferencing app
- Collaborative text editor
- Test script template
Why do it:
Usability tests allow you to validate your assumptions about the functionality of your product. Testing will help you to answer the question “how might the user interact with the design?” as well as “can the user accomplish their goals?”
When to do it:
When your prototype’s hypothesis is testable, prepare a usability test session. During the course of iterative design, this will most likely happen a handful of times. It’s perfectly acceptable to test an incomplete prototype or test lower fidelity designs.
Designers, stakeholders, and recruited users who match your persona profiles.
1 hour - script writing 1 hour - facilitating the test
One way to do it:
Start by identifying what you want to test. You need to decide a) what is the user’s goal and b) how might we create a way to test that goal. This can be done through real time time collaboration in a video conference, using user personas, journey maps, and flows as support material.
This results in defining a hypothesis that should be documented in the design documentation and github issue.
- User goal: Jane wants to reach out to an app developer about their product listing
- Hypothesis: We believe that creating a gallery of app listings will enable Jane to find an app and then reach out to its developer.
With your hypothesis (or hypotheses) in hand, prepare your testable prototype. Consider what fidelity you need to deliver for the test. If you are testing your assumptions for a user flow, a clickable wireframe, or InVision prototype it might be a faster test for you to conduct than a fully coded and polished application.
Write a script for the yourself or any additional usability test facilitators to follow. Script writing is a good opportunity to engage with and set expectations for stakeholders. These stakeholders include the product owner, project manager, the engineers, and designers. Use a collaborative writing tool like Google Docs so that all stakeholders can be involved in the writing; just make sure that everyone has editing rights!
There are many great templates for writing a script, including Steve Krug’s Usability Script. The template will help you structure the scenario that you need to create in order for participants to test your hypothesis. Now is a good time to refer back to those hypothesis statements and user flows. After a few asynchronous iterations, hold a video conference call so that you can do a verbal and visual walkthrough and identify any lurking concerns. When this is done, put the script in a shared location such as Google Drive or a GitHub wiki.
Schedule the test days. The test schedule should reflect the amount of time that is needed to do diligent research on the hypothesis. Avoid a situation where you have only one day for both testing and synthesis.
Recruit a handful of testers for the test days. Any given usability should involve nine or fewer users - but many have found that the sweet spot is five participants. Work with the product owner to recruit the appropriate participants for testing. The profile of the participants should match the profile of the users that you identified in your persona (the same users who you used to design the user flows or journey maps for). Whenever possible, compensate the participant for their time.
If you are in charge of recruiting, there are many templates and guides out in the world to help you frame the study. None of the test participants should be part of the core development team.
Before actually performing the test with participants, do a practice round with the internal product team. One way to do this is to pair designers up with developers and have each group set up a video conference. Both participants should try giving the test so that they uncover any issues that need to be addressed in the script or prototype - and frankly, to shake off any nerves before the day of the test.
Now’s the time for the test. Try to keep the ratio of participants to facilitators as close as possible. It often helps to have two people performing the test - one to facilitate and the other to observe and take notes. Constantly remember and remind participants and facilitators that you are testing the hypothesis, not the participant.
Analyze and synthesize the usability test results. Take a close look for any areas where the participant struggled with the task at hand and conversely, what worked really well (and shouldn’t be changed). These observations should be synthesized into a written summary that is communicated to the product team and stakeholders with a video conference call. A call is preferred over email so that you can have informal conversations about gut reactions, surprises, and successes. The informality of a video call can help you to maintain the iterative design spirit, which is still needed since you most likely will be troubleshooting bugs.
Document the next steps, most likely in the form of issues in the place where you track issues.
Test again and again and again… testing is all about assumption validation and should be repeated often.