Tree Testing

About

Tree testing is a research technique used on websites in order to understand the topic hierarchies a user perceives, and to evaluate content findability, with a goal of optimising that website’s information architecture (O’Brien D. 2009). It can be both a quantitative and qualitative method of research.

Tree testing is also known as reverse card sorting or card-based classification, task-based category testing, Information-architecture testing and taxonomy testing.

Tree testing is not performed on the website itself. Only two elements of a site’s information architecture, content organisation and labeling, are tested. Tree testing is typically used to evaluate an existing information architecture, or to test a new one.

By focusing only on the the information architecture, contextual and design-related content is removed from the study. (Le T., Chaudhuri S., Chung J., Thompson H. J., Demiris G. 2013)

Methodology

1. Planning – define the research goals, and what level of participation is required
2. Prepare trees – producing several different trees to be compared, this is known as “going wide”  (O’Brien D. 2016)
3. Define the tasks – usually the most frequently used, and critical activities for users. And,preferably less than 10 tasks – familiarity with the tree increases with higher task numbers (Optimal Workshop 2017)
4. Set up the test
5. Recruit participants
6. Pilot and run the test – response rates, initial scores, and drop-out rates should be closely monitored
7. Analyze the results – focusing on success rates, backtracking, slow response times, and patterns across tasks
8. Revise tree and retest – analysis will highlight the best-performing tree, and any issues that need improving. After these issues have been addressed, retests should be performed, and repeated, until a high-performing tree emerges – this type of retesting on one tree is known as “going deep” (O’Brien D. 2016)

A tree test would proceed as follows:

1. The participant is given a task, typically to find content, a product, or to locate a page
2. They are shown a list of the top-level topics of the website
3. They choose one of these topics, and are then shown a list of the subtopics for their topic choice
4. They continue choosing topics, moving through the tree, in any direction, until they find a topic that they feel satisfies the task, or until they give up.
5. This process is repeated for each task

Benefits

Benefits of Tree Testing include:

• It tests information architecture only – by being less ambiguous, results are more focused
• Tree tests can be created, run, and analysed quickly
• Only the tree of topics is used, a prototype isn’t required
• Tree testing enables researchers test multiple levels of the information hierarchy
• Because it is a task-based test, it better replicates users navigating a site than other information organising tests
• Research has shown that users who get the first click right have an 87% chance to complete the task successfully (Bailey B. 2013)

Limitations

• For lab-based tests, researchers and participants need to be together, which can either limit geographical diversity of test subjects, or require travel.
• Because tree testing only is carried out on the labelling, with no reference to the interface design, it doesn’t consider the influence of other information scents, such as iconography, which may affect findability

Case study

Tree testing of hierarchical menu structures for health applications

University of Washington

http://www.sciencedirect.com/science/article/pii/S1532046414000483

Introduction

Health Information Technologies (HIT) are highly complex systems, with interfaces that incorporate multiple menu labels, navigating across both broad and deep structures. (Le T., Chaudhuri S., Chung J., Thompson H. J., Demiris G. 2013)

In this case study tree tests are used to evaluate and optimise navigation and information architecture.

Critical Analysis

1. Tasks and scenarios were defined, based on discussions with the research team, they were chosen by merit of how they span the tree, and difficulty – ideally tasks would be based on the most critical, or most frequently used tasks
2. During the in-person sessions (online sessions were also held), participants were asked to think aloud as they performed tasks. This thinking aloud aspect of the in-person tests could negatively affect participants’ concentration and performance, compared to those remotely tested
3. Participants were recruited primarily from the university population – this younger, educated subject may not have been a good map to the target demographic
4. Because testing was performed both in person and online, test conditions would not have been consistent, environmental factors, and technology factors could affect success rates and response times

References

Cardello J. (2014) Low Findability and Discoverability: Four Testing Methods to Identify the Causes. Retrieved February 3, 2017 from https://www.nngroup.com/articles/navigation-ia-tests/

O’Brien D. (2009, December 5). Tree Testing: A quick way to evaluate your IA. Retrieved February 2, 2017 from http://boxesandarrows.com/tree-testing/

O’Brien D. (2016) The design phase: going wide Retrieved February 2, 2017 from https://treetesting.atlassian.net/wiki/display/TTFW/The+design+phase%3A+going+wide

O’Brien D. (2016) The design phase: going deep Retrieved February 2, 2017 from https://treetesting.atlassian.net/wiki/display/TTFW/The+design+phase%3A+going+deep

Le T., Chaudhuri S., Chung J., Thompson H. J. Demiris G. (2013, November 13) Tree testing of hierarchical menu structures for health applications Retrieved February 2, 2017 from http://www.sciencedirect.com/science/article/pii/S1532046414000483

Optima Workshop How to write effective tasks for your tree tests Retrieved February 1, 2017 from https://support.optimalworkshop.com/hc/en-us/articles/204442140-How-to-write-effective-tasks-for-your-tree-tests