Write a UX survey

Outcome

A UX survey of about five to ten questions, ready to send. Each question is tied to a decision someone will make with the answers. The rating questions share one scale, the wording is neutral, and there is at most one or two optional open questions aimed at the people whose answers need explaining. The survey has been tried by at least three people before launch.

When to use it

• You need a number you can compare over time, such as how easy people find a task, release after release.
• You want to know how common a problem is that you have already seen in interviews or usability tests.
• You can reach enough people who recently used the part of the product you are asking about.

When not to use it

• You do not yet know what the problems are. A survey counts things you already know to ask about; run interviews first.
• You want to know why people behave as they do. Most of what you need would be open questions, which is an interview study.
• You want to know whether people can complete a task. Watch them try it in a usability test.

Before you start

• The decisions. A short list of what the team will decide with the results, and who will read them.
• The audience. Who should answer, and how you will reach them at the right moment, such as straight after a task.
• A tool. The UX Survey Template for a tested set of statements, the Survey Questions bank, and the Survey Builder to assemble and preview the form.
• Time. Half a day to write and an hour to pilot.

Steps

Step 1: Write the decisions the survey will inform

List what the team will do differently depending on the answers, and who reads them. For every question you write later, you should be able to point to one of these decisions.

You should now have: a list of decisions, each with the person who will act on it.

Step 2: Draft only the questions that serve a decision

For each decision, write the smallest set of questions that would inform it. Delete any question you would merely find interesting. Keep the whole survey to about ten questions or fewer; long surveys lose respondents and the late answers get careless.

You should now have: a draft of five to ten questions, each mapped to a decision.

Step 3: Choose one rating scale and use it everywhere

Use one scale for all rating questions, such as a five-point agreement scale from strongly disagree to strongly agree. Label every point, not only the ends. Keep the direction the same in every question so the answers compare.

You should now have: every rating question on the same labelled scale, in the same direction.

Step 4: Remove leading and double-barrelled wording

Read each question and ask whether it suggests an answer, assumes something, or asks two things at once. How much did you enjoy the new dashboard? assumes enjoyment; The dashboard showed me what I needed is a neutral statement to rate. Split any question with and in it.

You should now have: questions that each ask one thing without suggesting an answer.

Step 5: Add one targeted open question

Add one optional open question that explains a rating, shown only to the people whose rating needs explaining, such as What got in the way? for anyone who rated a task 1 or 2. Ask about a specific moment, not general comments. Keep it off the first screen.

You should now have: at most one or two optional open questions, each tied to a rating and a subgroup.

Step 6: Order the questions and build the form

Put easy, relevant questions first and anything personal last. Group questions about the same task together. Build the form in your survey tool and check that it works on a phone.

You should now have: a working form in a sensible order that reads well on a phone.

Step 7: Pilot it with three people

Ask three people from the audience to fill it in while thinking aloud, or at least to tell you afterwards which question they hesitated on. Fix any question they misread, and time how long it takes.

You should now have: a piloted survey with confusing questions fixed and a known completion time.

Worked example

A project management tool has redesigned how tasks are assigned. The team wants to know whether assigning got easier and for whom it did not.

1. Decisions: keep or roll back the new assign menu (product lead); which team sizes need follow-up interviews (researcher).
2. Six questions survive: two about assigning, one about finding tasks assigned to you, team size, role, and an open question.
3. Scale: five points, strongly disagree to strongly agree, every point labelled.
4. Wasn't the new assign menu much faster? becomes Assigning a task took little effort. The menu was clear and fast is split into two statements.
5. Anyone who rates Assigning a task took little effort at 1 or 2 sees: What happened the last time assigning a task was hard?
6. The survey is triggered after someone assigns a task, with team size and role at the end.
7. In the pilot, two of three people read "assign" as "assign to myself". The statement becomes Assigning a task to a teammate took little effort. The survey takes ninety seconds.

Checklist

• Every question maps to a decision with a named reader.
• The survey has about ten questions or fewer.
• All rating questions use one labelled scale in one direction.
• No question suggests an answer or asks two things at once.
• Open questions are optional, targeted at a subgroup and off the first screen.
• The form works on a phone.
• Three people from the audience have piloted it and the confusing questions are fixed.

Common mistakes

• Asking what would be interesting. Every extra question costs answers on the questions that matter.
• Mixing scales. Switching between five and seven points, or flipping the direction, makes answers impossible to compare and confuses respondents.
• Asking people to predict their behaviour. Would you use this feature? measures politeness. Ask about what they did.
• Ending with an open "any other comments?" box. It collects little, and nobody is assigned to read it.
• Skipping the pilot. A misread question is found by three people in an hour, or by the whole sample after launch.

Related

• UX Survey Template: rating statements about usability and clarity, ready to send.
• Survey Questions: a bank of neutral questions to pick from.
• Survey Builder: assemble the questions and preview the form.
• Open Ended Questions vs Closed Questions in Surveys: when each kind of question earns its place.