Understanding HCI

Co-discovery Learning

What is Co-discovery Learning?

Co-discovery Learning (CL) is an adaptation of the most commonly used type of user testing - think aloud protocol (TA). In CL, users are grouped in pairs and talk aloud naturally to each other whilst competing a task. Talk aloud facilitates the evaluators in that the users train of thought can be followed and erroneous assumptions about the system can be noted.

They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product. They are encouraged to explain what they are thinking about while working on the tasks. Compared to thinking-aloud protocol, this technique makes it more natural for the test users to verbalize their thoughts during the test.

Why Use it?

Co-discovery Learning is more realistic than a single user scenario, as people in work environments often work in teams. The users often find it easier and more natural to vocalise thoughts with a colleague present. The evaluators can also quantify the time taken for various tasks, the number of tasks competed correctly, the error frequency, numbers of times the users accessed the help system etc. (Doubleday et al, 1997, p.104). From this they can make more qualitative judgements such as the success or lack of the entire system; system sub-components; help system; effort required to achieve a particular result; quality of interface.

Participants Needed

Experts

1 usability expert and 1 camera operator is required for the exercise.

Users

A minimum of 6 users should be observed - the more users that can be observed, the better the results. Different Users will have different problems. The number of users must be even since pairs of users are observed.

Task List

  1. Pair the test users into groups of two. It is preferrable to pair two users who know each other into one group so that they won't feel uncomfortabel working together.
  2. Provide the test users with the product to be tested (or a prototype of its interface) and a scenario of tasks to perform.
  3. Tell the users that it is the system that is being evaluated not them.
  4. Ask users to sign a consent form allowing you to videotape and use the data.
  5. Ask them to perform the tasks using the product, and explain what they're thinking about while working with the product's interface.
  6. Gradually move from simple tasks to more difficult ones.
  7. Have them help each other in the same manner they would if they were working together to accomplish a common goal using the product.
  8. At the end, have them complete an evaluation form or memory test.
  9. Videotapes are analysed by the evaluator
  10. Solutions are incorporated into the prototype which is again retested.

Conditions required

  • Small quiet room (with LAN access?)
  • Passwords prepared
  • Dummy records and results prepared
  • Consent forms
  • Co-discovery Learning task sheets
  • Evaluation forms
  • Video recorder and operator
  • Observer

Examples

Limitations Of method

  1. This method is quite time intensive.
  2. Co-discovery Learning may not uncover all errors as users are doing only a subset of the full amount of tasks.
  3. Users may complete tasks in spite of problems or errors encountered either through persistence or learning.
  4. Some errors may be impossible to find by this method, such as poor window or navigation structure which is still usable but not optimum.
  5. User testing only uncovers symptoms of problems, not the cause – this must still be inferred by an expert.
  6. Using two people at a time means that is can be harder to find a large enough representative group to conduct the tests.

Exercise

Reading

  • J. Nielsen "Usability Engineering" p.198, Academic Press, 1993.
  • Dumas and Redish, A Practical Guide to Usability Testing, Ablex Publishing. p. 31, "Co-discovery".
  • Rubin, J. Handbook of Usability Testing. John Wiley & Sons. p. 240, "Testing two participants at a time."