Dependent Variables in UX Research: Choosing the Thing You Measure

A team redesigns a help center and picks time on page as its success metric. Time on page goes up. They call it a win and roll it out.

Time on page went up because people couldn't find the answer.

Nothing was wrong with their statistics. The problem sat one step earlier, in the choice of what to record. Whatever you measure becomes the definition of success for everyone reading the report afterward, and most people reading it will never revisit whether the definition was any good.

The dependent variable is where "better" gets defined

A dependent variable is the outcome you record — the number or category you expect to move when you change something. Task completion. Time to first successful export. Error count. A rating on a 1–7 ease item. If the independent variable is the thing you set, the dependent variable is the thing you're watching, and the split between them is worth having straight before anything else (independent vs dependent variables).

What gets less attention is that the choice is a value judgment wearing a methods costume. Pick conversion and you've decided the design's job is to get people through. Pick time-on-task and you've decided speed is the point, which is right for a support queue and wrong for a product people are meant to browse. Pick a satisfaction rating and you've decided the felt experience outranks whether anyone actually finished. Three teams can run the identical study on the identical prototype, choose three different dependent variables, and ship three different products — all correctly, given their measures.

So the real first question isn't "what can we measure?" It's: if this number moves, what will we do? If nobody can answer that, the measure is going in the study for the wrong reason.

Operationalize it until two observers agree

A measure isn't finished when you can name it. It's finished when two people watching the same session would write down the same value.

"Did they succeed" fails that test immediately. Succeed at what — reaching the confirmation screen, or reaching it without help, or reaching it inside two minutes? Someone who found the right page and abandoned at the form: success or not? If the answer is being decided during analysis, the measure is being decided during analysis, and by then the honest word for the study is exploratory.

Write the rule before collection, in this shape:

Task success = participant reaches the order-confirmation screen for the specified item, unaided, within 5 minutes. Partial completions and self-corrected wrong-item orders count as failures.

Ugly, and that's the point. NN/g's case for success rate as the simplest usability metric rests entirely on this: it's simple only after somebody has done the unglamorous work of drawing the boundary, and that work is where the disagreements surface. Better to have them in a planning doc than in the readout.

The same discipline applies to timing. Time to what — first click, first correct click, last click before the confirmation? Do you subtract the seconds the participant spent talking to the moderator? A think-aloud protocol inflates task time by a wide margin, so a study that times participants while asking them to narrate has a dependent variable measuring two things at once.

Check it for validity and reliability, then check it for sensitivity

Validity — does it measure the thing you care about? This is where time on page fails. It's a real number, precisely captured, and it points at "engagement" only if you assume reading equals satisfaction. For a help center the plausible reading is the opposite one.

Reliability — would you get the same value twice? Anything a human codes needs a rule tight enough that two coders land in the same place. If the second coder's numbers differ from the first's, the variation you're about to report as a finding is partly just the coders.

Sensitivity is the one that quietly ruins studies, and it gets less airtime than the other two because it isn't a property of the measure alone. It's a property of the measure against your population and your expected effect size. A binary success rate on a task nearly everyone already completes has almost no room to move. Ninety-something percent pass before the change, and the ceiling absorbs whatever improvement you made; you would need a very large sample for the remaining points to clear noise, and most teams have thirty participants and two weeks.

The design got better and the measure couldn't tell. The team read the flat number as evidence the change didn't matter, and moved on to something else.

When you expect that, switch to a graded measure before you collect: time, error count, number of steps, a rating scale. Those can register improvement inside the group that was already succeeding, which is where the improvement actually happened.

Behavioral and attitudinal measures answer different questions

What you record	Typical dependent variable	Answers
Behavior	Task success, time on task, error count, steps taken	Could they do it?
Attitude	SEQ, SUS, confidence rating, perceived difficulty	How did it feel?
Consequence	Return visits, support tickets filed, feature adoption at 30 days	Did it hold up outside the lab?

The rows disagree more often than teams expect, and the disagreement is usually the finding. People who completed the task in 40 seconds and rated it a 3 out of 7 got through something they didn't understand — that's a different problem from failing outright, and only visible because both were recorded. NN/g's comparison of perceived usability instruments is a useful map of the attitudinal column; the Single Ease Question is the cheapest entry in it, one item after each task, and it survives being asked eight times in a session in a way a full questionnaire does not.

Pairing one behavioral measure with one attitudinal measure, per task, is a reasonable default for most moderated work — including the step-by-step usability test most teams already run. The broader menu of when each belongs sits with the research methods that produce them.

One primary, a few guardrails

The temptation is to record everything and sort it out later. Instrumentation is cheap, and every extra measure feels like insurance.

It isn't. Run twenty dependent variables against one change and something clears significance by luck alone. Whichever one moved becomes the headline, and the report reads like a finding rather than the lottery result it is.

The structure that holds up:

• One primary dependent variable, named in writing before collection. This is the one the decision hangs on.
• Two or three guardrails — measures that must not get worse. Error rate, support contacts, time on a neighboring task. A guardrail can stop a launch. It can't justify one.
• Everything else is exploratory, and gets labeled that way in the report. Exploratory findings are hypotheses for the next study, not results.

Write the primary one down in advance, because hindsight is extremely good at making whichever number moved feel like the one you always meant. NN/g's argument for keeping A/B testing in its place leans on the same discipline. It isn't a large-sample precaution either; a five-person moderated session with four recorded measures has the same problem in miniature.

Four ways the measure breaks the study

The proxy drifts from the thing. Time on page for "engagement," clicks for "interest," scroll depth for "read it." Every proxy is a bet that the correlation holds under the change you're making. A redesign is the event most likely to break it. Confusion and engagement produce the same trace.

The measure only exists for one group. Recording "time from signup to first project" excludes everyone who never made a project, which is the group with the problem. The dependent variable is defined on survivors, and the result describes people who were already fine.

Definition moves mid-study. Someone decides in week two that partial completions should count. Now weeks one and three aren't comparable, and the fix — recoding everything under one rule — is only available if the raw sessions were kept.

It's collected from a leading question. "How easy was that to find?" and "Did you have any trouble finding that?" don't produce the same ratings, and neither survives being asked by the person who designed the screen. The wording of the instrument is part of the measure (task scenarios).

A four-line spec before you run anything

Fill these in and most of the failures above become visible while they're still cheap:

1. Primary measure: what, in what unit, at what moment, with what counts as failure.
2. Why it's valid: the sentence connecting it to the thing you actually care about. If the sentence needs an "assuming that," write the assumption down.
3. Guardrails: the two or three numbers that must not get worse.
4. Decision rule: what you do if it moves, and what you do if it doesn't.

Line four is the one that gets skipped, and it's the one that catches a dead measure. Write out what you'll do if the number moves, then write out what you'll do if it doesn't. If both answers are the same action, go back to line one. Something in the study is being measured for reassurance rather than for a decision.

Frequently asked questions

What is a dependent variable in UX research?

The outcome you record to see whether a change mattered — task success, time on task, error count, a satisfaction rating. It's called dependent because you're testing whether it depends on the thing you changed.

Can a study have more than one dependent variable?

Yes. Most do. Only one of them can be primary, and that one gets named before collection starts.

Is a survey rating a dependent variable?

It is, if it's the outcome you're comparing across conditions. A post-task ease rating in an A/B comparison qualifies. The same rating collected once, with nothing to compare it against, is a benchmark you can track over time, which is a useful thing to have and not a study result.

How do I choose between task success and time on task?

Ask what failure looks like in your product. If people get stuck, success rate will move. If nearly everyone finishes and the complaint is that it's a slog, success rate is already at the ceiling and time is where a change will show up (usability metrics).

What's the difference between a dependent variable and a KPI?

Scope. A KPI is a standing business measure someone watches continuously. A dependent variable is picked for one study to answer one question, which usually makes it far more specific than anything on a dashboard would be. "Time to first successful export, unaided, excluding sessions where the moderator intervened" is not a KPI and was never meant to be.

Do qualitative studies have dependent variables?

Not in the strict sense. A study with six participants and no comparison condition isn't measuring an outcome across levels, and forcing a count onto it ("4 of 6 struggled") lends arithmetic weight the design can't support. Report what you saw and why it happened. The dependent variable belongs to the study you run afterward, once there's something to compare against.