Watching people use your product is the most effective way to ensure it best meets their needs. While observing sounds like an easy thing to do, there’s more to usability testing methodology than meets the eye. These tips from our work at Centralis can help you go “behind the scenes” to understand the facilitator’s approach, interpret participants’ behavior accurately, and get the most out of the sessions you watch.

1. Be a fly on the wall

Join the web conference anonymously, keep your camera off, and ensure you are muted for the entire session. If you're observing in person, position yourself out of the participant's view so that you don't intimidate them.

2. Minimize distractions

When watching sessions remotely, it’s easy to get distracted by email notifications, Slack messages, etc. Strive to focus your full attention on watching the session; otherwise, you may miss important details and draw incorrect conclusions about what’s happening in the research.

3. Be familiar with the tasks

We ask participants to complete tasks so we can understand how the product works from their perspective. Each task is carefully chosen to explore specific research questions and elements of the design. If you review the discussion guide prior to the sessions, you’ll have a good sense of what the tasks are and why they’ve been included.

4. Pay attention to what people do, don’t just listen to what they say

Participants will “think aloud” while completing tasks to help us see what the product feels like through their eyes. That said, it’s critical to watch their behavior, too. People aren’t always aware of the difficulty they’re having, they want to please the facilitator, and they don’t want to look stupid. It’s common for a participant to struggle with or fail a task, and then declare, “That was easy!” Their behavior is a better measure of their experience than anything they might say about it afterwards.

5. Be patient

The facilitator will let the participant attempt to complete the task on their own, even if they are having great difficulty. Watching people struggle is hard, but it’s how we learn what needs to change to make the product better. It’s common for the facilitator to ask the participant questions that we already know the answers to; this is to explore the user’s understanding further.

6. Watch for:

a. Hesitations – They may pause a minute before taking an action. The facilitator may ask, “What are you thinking?” to explore the hesitation.

b. Expectation mismatches – Exclamations like “Oh!” or “Um, ok…” may indicate that an action didn’t have the anticipated result. The facilitator may ask “Is this what you were expecting?” to learn more about the mismatch.

c. Backtracking – Using the back button, retracing steps, and returning to the start screen are all signs that the user may be lost and trying to reorient themselves.

d. Frustration indicators – Participants who are frustrated may sigh, raise their voices, or say things like, “I would be done with this!” Note when these behaviors occur, as they often reveal “last straw” issues in the interaction. The facilitator will gently ask the participant to explain their frustration.

7. Focus your notes on what is happening, not what it might mean (yet)

In any given session, it’s hard to know what might be significant later, so if you’re taking notes, record observations rather than potential solutions. For example, you might make a note that, “user doesn’t interact with menu” rather than “make the menu button red”. There will be time to solve the problems once all the findings are in.

8. Look for patterns

Don’t jump to conclusions based on the first person or two. Strive to watch each session with fresh eyes until enough are completed that patterns are emerging.

9. Avoid common traps:

When watching someone struggle with your product, it’s tempting to dismiss their feedback by falling into these common traps:

a. Blaming the user – Usability problems are never the user’s fault – if people can’t use your product, they just won’t, and your business will suffer. Keep an open mind and a customer service mindset to get the most out of your research.

b. Discrediting the user – We go to great lengths to recruit participants who are representative of your target audience. They may do or say things that you don’t want to see or hear, but that doesn’t mean they’re not your target users.

c. Making excuses – Every product is subject to business constraints that limit what can be provided and how. Be sure you’re acknowledging those constraints without using them as a reason to dismiss the user’s feedback. Usability testing can often provide the justification for overcoming business issues in the name of providing a better user experience.

10. Stay positive!

Every product has usability issues. In over 20 years of research, we’ve never had a single session that didn’t identify something that could be improved. While it can be hard to watch people get frustrated with your project, the point of research is to learn how to make things better. It’s always better to know about problems because then they can be fixed.