Open Ended Questions vs Closed Questions in Surveys

A team runs a satisfaction survey and puts one free-text box at the bottom: Anything else you'd like to tell us? A couple of thousand people answer it. Someone opens the sheet, reads the first screenful, finds three comments that confirm what the team already believed, pastes them into a slide under the heading "In their own words," and closes the tab.

The question was badly placed: at the end of a survey nobody had budgeted time to analyze, aimed at everyone, about nothing in particular. The cost of an open question isn't paid when you write it. It's paid weeks later, by whoever has to turn a column of prose into a sentence a team can act on.

That is the trade underneath open ended questions vs closed questions, and most survey guides describe a different one. Both kinds are easy to write. What differs is the bill: an open question is cheap to answer and expensive to read.

What each kind of question actually is

A closed question fixes the answer set before anyone sees it. Yes/no, single-select, multi-select, a rating scale — the respondent picks from a menu you wrote. Every answer arrives as a value in a column, so the analysis is arithmetic.

An open question leaves the answer set undefined. The respondent types whatever they want, in whatever length and register they want, and what comes back is text. Nothing is countable until a person reads it and decides what the categories are.

The distinction is who defines the categories, and when. With a closed question you commit up front, and you're stuck with those categories whether or not they were right. With an open question you defer the decision and pay for the deferral in reading time.

What only open questions can do

Three jobs, worth naming precisely because so many surveys ask open questions for none of them.

Find the vocabulary you don't have. People describe their work in words that are not your feature names — they call the thing you named "Workspaces" a folder, or a project, or "the place where the files are," and which one they say tells you what they think it's for. You cannot write a closed question about a problem you can't name yet, and a menu of your own terms will confirm only that your terms exist. Everything downstream depends on this: what you call the feature, what you write in the empty state, which problem you decide is worth solving at all. It's the strongest reason to run open ended questions early, before there's anything to count.

Catch the answer that isn't on your list. Every closed question assumes its options cover the space. When 15% of a multi-select goes to "Other," the open box beside it is the only thing that tells you what "Other" was.

Explain a number you already have. A satisfaction score drops four points and nobody knows why. A targeted open question, asked only of the people who dropped, is the cheapest instrument that exists for that. Note the constraints in that sentence: targeted, and only of the people who.

What only closed questions can do

Count reliably and compare across time. If you want to know whether satisfaction moved since last quarter, you need the same fixed answer set both times. Free text can't do that. Two readers will code the same 400 comments differently, and you won't be able to tell a real change from a change in whoever did the reading.

Closed questions also cost the respondent almost nothing, which matters more than it sounds. Every open box raises the effort of finishing the survey, and effort is what produces abandonment, and what produces one-word answers from the people who do finish. Three open questions in a row and you're collecting "n/a", "none", "good" from everyone except the unusually motivated.

The mechanics of building those fixed answer sets well — the forms, the response-option traps, the pretest — are a subject of their own, covered in writing closed questions for research and, for rating scales specifically, the Likert scale guide.

Decide by what happens after the survey closes

One rule does most of the work here: ask an open question only when you can name the person who will read every answer, and the week they'll do it.

If you can't, don't ask it. Not "ask it anyway, it's free": it isn't free. An unread open question costs you respondent effort, response rate, and the specific illusion that you have qualitative evidence when what you have is an unread column. Unread text is worse than no text, because it looks like coverage.

Run the numbers before you write the question. Time yourself reading and coding twenty real responses, then multiply by the sample you expect. Coding runs a lot slower than skimming — call it half an hour for those twenty — which puts a couple of thousand answers at the better part of a working day, and that's the first pass, before anyone reconciles two coders' categories. For a 200-person study the same question is an hour's work and entirely reasonable.

Placement matters more than the count. One open question asked of a filtered subgroup — "you selected 'Other' — what was it?" — produces maybe 60 answers, all about one thing, and gets read. The same question asked of everyone at the end of the form produces two thousand answers about everything.

Open ended questions examples, before and after

The common failure is asking one so broad that the answers can't be compared to each other.

Before: Anything else you'd like to tell us? After: Thinking about the last time you exported a report — what were you trying to do with the file after you downloaded it?

The first invites anything, so it returns everything: praise, a bug report, a pricing complaint, a blank. The second fixes a moment, a task, and a scope, and leaves the answer open. Every response lands in the same territory, so the responses can be read against each other.

Before: How could we improve the dashboard? After: What did you check on the dashboard this morning, and where did you look first?

"How could we improve" asks the respondent to do your design job with none of your context, and it reliably returns feature requests that are guesses. Asking about a specific past action returns behavior. Same instinct as a well-moderated interview: ask about what happened.

Before: Why did you cancel? After: What was going on the week you decided to cancel?

"Why" invites a tidy retroactive rationalization. People supply the reason they think you want, or the reason that makes them look consistent. Asking about the week returns the circumstances, and the circumstances are usually where the cause lives.

Mixing them in one survey

The pairing that works is a closed question that measures and an optional open question that explains, shown only to the segment whose answer needs explaining. Rate the experience, then — for anyone who rated it 1 or 2 — one open box asking what specifically got in the way. You get a countable number and a small, readable pile of text about that number.

Keep the open boxes optional, or you'll get "n/a" from people who had nothing to say and would rather be done. And keep them off the first screen; an open box early in a form loses respondents before you've collected anything.

In an interview, the ratio inverts

Everything above is about surveys, where a question is asked once by a form that can't follow up. In a moderated session that constraint disappears. You're there. You can ask what they meant. Open questions carry nearly the whole session, and closed ones exist mostly to open a door — do you use it weekly? — before you walk through it with "tell me about the last time."

Which means the survey-versus-interview choice usually comes first. If most of what you need is open ended questions, you may be looking at a study that should have been user interviews all along: twelve conversations you can probe rather than two thousand text boxes.

What to change on the next survey

Go through the draft and, for every open question, answer two things in writing: who reads these, and by when. Delete the ones with no answer. For the survivors, check that each names a specific moment or task rather than inviting general comment, and that it's shown to the subgroup it's actually about. Then count what's left. More than two on a form a stranger is expected to finish, cut to two.

For everything you cut, ask whether the same thing can be learned from a closed question with a well-built answer set, or whether it was never a survey question at all. Most of what gets shoved into a free-text box at the end of a form is a research question that deserved its own study, which is what the rest of the research methods collection is for.

FAQ

What is the difference between open and closed questions?

A closed question fixes the possible answers in advance and returns a value you can count. An open question leaves the answer undefined and returns text, which has to be read and categorized by a person before it means anything. The difference is when the categories get defined — before the survey, or after it.

When should I use open ended questions in a survey?

When you don't yet know the vocabulary or the answer set, when you need to explain a number a closed question already produced, or when you're catching what an "Other" option missed. And only when someone is actually scheduled to read the answers.

How many open questions should a survey have?

For a general-audience survey, two at most, both optional, neither on the first screen. Targeted follow-ups shown only to a filtered subgroup are cheaper and can be more numerous, because each one reaches far fewer people.

What is an example of an open ended question?

Thinking about the last time you exported a report — what were you trying to do with the file after you downloaded it? It fixes a moment and a task while leaving the answer open, so the responses can be read against each other instead of scattering across unrelated topics.

Are open questions better than closed questions?

Open questions are over-asked and under-read, so in most surveys the honest answer is that you should be asking fewer of them. The exception is real and narrow: when you can't name the categories yet, a closed question will only confirm the ones you guessed. And when you need to compare a result to last quarter's, only a closed question will do it.

Do open questions lower survey response rates?

They raise the effort of completing a form, which shows up as both abandonment and thinner answers from the people who finish. Keeping them optional, few, and late in the form limits the damage.

How do you analyze answers to open questions?

Read a sample first and draft categories from it, then code the full set against those categories, adding new ones as you hit answers that don't fit. Have a second person code a slice independently and compare. Where you disagree is usually where the category was vague.