5 Questions and Answers about Remote UX Testing and COVID-19

As the tech sector scrambles to ensure that life can continue almost exclusively online, it’s as important as ever for users to be part of the design process. Fortunately, UX professionals have been leveraging the Internet to reach users worldwide for many years now. In fact, over the past two years at Centralis, 60% of our user research engagements have included remote participants, and close to 100% of projects have featured remote client observers.

Centralis is fortunate in that remote UX research is what we do, every day, pandemic or no. We’ve compiled a short list of questions and answers to help product teams see how they can bring their users into the design process, even when everyone is stuck at home:

1. How does remote usability testing work?

Moderated remote testing works exactly like in-person testing, just using web conference software. The participant and the researcher both join a call using software like Zoom, GoToMeeting, or WebEx. (Stakeholders and other observers can join too.) After a brief introduction to explain the setup and establish rapport, the researcher shares their screen with the participant, gives them control of it, and asks them to complete tasks while thinking aloud.

The research team observes the participant’s actions, hears their commentary, and can even see their facial expressions (if webcams are on). We ask probing questions to explore the participant’s understanding of the interface, just as if we were sitting beside them. We identify and diagnose any usability problems in the design. Based on these insights, we provide concrete, actionable recommendations for improving the product’s user experience - all without anyone leaving their homes.

2. What can I test online?

Just about anything - if you can put it on a screen, you can test it with users remotely. Most of the remote testing we do is with digital interfaces, such as websites, software, or mobile apps. Typically, the participant interacts with the researcher’s computer - this protects their privacy and allows us to show prototypes of works-in-progress that aren’t yet live.

It’s common for us to share a prototype of a early-stage design built using a tool like Axure, Invision, or Marvelapp - these platforms allow you to test realistic interactions before you invest the time and effort in coding. If you’re looking for early reactions to ideas, you can even show sketches or concept statements. While physical products may seem a bit more challenging to test remotely, that’s possible too: the participant just needs to show you what they’re doing through the webcam.

3. Should the sessions be moderated?

As an alternative to moderated sessions, many online testing platforms enable you to set up a test that participants can take on their own, without interacting with a researcher. Some of these services will record the sessions for the team to watch later, while others simply collect survey responses from participants after they complete the tasks. These options can be useful if you’re looking to gather benchmark data or compare two designs, but in general, they tell you less about users’ motivations, behavior, and sources of confusion because you can’t ask follow up questions or get clarification. If you’re interested in learning more about different types of remote UX testing, check out this post from Centralis’ blog.

The best approach for remote usability testing depends on what you’re hoping to learn. Use moderated sessions to identify where users may have difficulty, and why. Participants in moderated sessions are motivated to persist with a task because the researcher is right there with them. The researcher can also formulate and test hypotheses about the nature of the participant’s trouble, which is critical for pinpointing the cause of the problem in the design.

4. How many participants do I need?

As with any qualitative research method, a little goes a long way. For most interfaces, you can learn what you need to know at a given point in time by running sessions with 6-8 people for each user role.

For example, we might want to test a hospital system’s website with 12 users: 6 patients and 6 healthcare professionals. We often over-recruit by one or two to guard against technical difficulties or unanticipated interruptions, so we would look to schedule up to 8 people in each role with a goal of completing at least 12 sessions overall.

Iterative testing is always preferable to a single, larger, sample: It’s always better to learn from 6-8 users, update the design to fix the issues identified, and then test again. You can learn more about determining the best sample size by reading Is Eight Enough? Choosing the Right Sample Size — Centralis.

5. How long does remote usability testing take?

The key driver of timing in a usability testing project is the ease or difficulty of recruiting your target audience. For a general consumer audience, you can schedule the sessions a few days or a week in advance. We generally run 5-6 sessions per day, followed by a debrief - so it’s possible to have test findings in hand within a week or two.

If your target audience is more specific; for instance, professionals like CEO’s, corporate counsel, or cardiologists, a lead time of two to three weeks for recruiting is more realistic. Sessions may also need to be scheduled more flexibly to account for the participants’ own busy schedules. We still move quickly to debrief on the findings; in this scenario, you can often know what you need to know from the research within three to four weeks.

We hope this Q&A has helped illustrate that, even during a pandemic, it’s still possible to bring the voice of the user into the design process. Some of us work on interfaces that involve life-and-death situations, where usability issues can be fatal. For the rest of us, however, the stakes are also high: in a time of unprecedented stress and uncertainty, we have the power to find and eliminate frustrations that could feel to users like the last straw. Remote usability testing is a relatively quick and easy way to eliminate all that unnecessary unhappiness and risk. If there was ever a time where we need our products to be clear and easy to use, it’s now.

To learn more about remote UX testing, please visit centralis.com/remote-usability-testing.