Run a tree test

Outcome

For each task, a success rate, a directness rate and the path most people took when they went wrong. From those, a short list of labels to rename and items to move, and a revised tree ready for the next round.

When to use it

• You have a draft structure, from a card sort or a redesign, and want to know whether people can find things in it.
• You want to compare the current navigation with a proposed one using the same tasks.
• The team argues about where an item belongs and wants evidence instead.

When not to use it

• You do not have a structure yet. Start with a card sort.
• The problem is in the visual design of the menu, such as hidden items or poor contrast. A tree test removes the visual design on purpose.
• The structure is shallow and short, such as five items on one level. There is not enough to get lost in.

Before you start

• The tree. The full hierarchy as text, every level, using the exact labels you plan to ship.
• Participants. At least 30 people who would use the navigation, if you want success rates you can compare between versions.
• The tool. A tree testing tool, such as the Tree Test in Pulse.
• The items that matter. The content people most often look for, from search logs, support tickets or analytics.

Steps

Step 1: Enter the tree as plain text

Write the hierarchy in the tool exactly as it will appear: every label, every level, no descriptions or icons. Include the parts you are not testing, because people may look there.

You should now have: the whole tree in the tool, with the exact labels and levels.

Step 2: Choose what to test and mark the correct answers

Pick eight to ten items people need to find, starting with the most important ones and the ones the team disagrees about. For each, mark every location in the tree that counts as correct.

You should now have: eight to ten target items, each with every correct location marked.

Step 3: Write each task as a need, without the label

Write each task as a situation, such as you were charged twice this month; where would you go to sort it out? Do not use the words of the label you are testing. A task that contains the answer measures word matching.

You should now have: eight to ten tasks written as situations, none containing the label they test.

Step 4: Pilot with two people and fix the tasks

Run the test with two people who fit the profile. Check for tasks they misread and for tasks with more than one fair answer you had not marked. Rewrite those tasks or add the missing correct locations.

You should now have: piloted tasks with every fair answer marked.

Step 5: Collect results from at least 30 people

Send the test, randomise the task order and keep each person to eight to ten tasks. Close it once at least 30 people in the profile have completed it.

You should now have: completed tests from at least 30 people, with tasks in random order.

Step 6: Read success, directness and the first click per task

For each task, note the success rate, the share who went straight to the answer without backtracking, and where people clicked first. A low first-click rate on the right branch means the top-level label is the problem; a right first click that ends in failure means the problem is lower down.

You should now have: for each task, the success rate, directness and where people went first, with the level where it fails.

Step 7: Decide the changes and retest

For each failing task, decide one change: rename a label, move an item, or list it in a second place. Make the changes in a copy of the tree and run the same tasks again with new participants.

You should now have: a revised tree with a written reason for each change, and a second round planned with the same tasks.

Worked example

A council website is replacing a navigation organised by department with one organised by what residents need to do.

1. The new tree has six top-level sections and 74 items.
2. Nine targets, including report a missed bin collection, apply for a parking permit and pay council tax.
3. One task reads: your bins were not emptied on Tuesday; where would you tell the council? None of the tasks contain report, bin or collection as labels.
4. In the pilot, one person looks for parking permits under Roads and transport and Apply for something; both are fair, so both are marked correct.
5. 41 residents complete the test.
6. Most tasks score above 70%. Report a missed bin collection scores 38%: 52% clicked first into Rubbish and recycling but then could not find it, because the item sits under Report a problem.
7. The item is listed in both places, and the retest a week later scores 81% on the same task.

Checklist

• The whole tree is entered as text with the exact labels.
• Eight to ten tasks cover the most important items and the disputed ones.
• Every correct location is marked for each task.
• No task contains the label it is testing.
• The test was piloted with two people and fixed.
• At least 30 people in the profile completed it, with tasks randomised.
• Each failing task has a decided change and a reason.
• The revised tree is retested with the same tasks.

Common mistakes

• Giving the answer in the task. "Find the billing settings" tests whether people can spot the word billing.
• Testing only part of the tree. If the rest is missing, people cannot go to the wrong place, and the success rates look better than they are.
• Marking one correct answer. If a reasonable person could look in two places, both are right. Missing one makes the structure look worse than it is.
• Reading success alone. A task can succeed after a long detour. Directness and first clicks show where the structure misleads.
• Fixing without retesting. A renamed label can fix one task and break another.

Related

• Tree Test: run the tasks against your tree with the panel and see success rate, directness and paths.
• Card Sort: build the draft structure the tree test checks.
• Run a card sort: the step before a tree test.
• Recognition rather than recall: why people should recognise their task in a label.
• Information architecture and card sorting course: the method in lessons with knowledge checks.
• Designing Intranet Information Architecture: why to test the tree before building the pages.