GR6: User Testing

Find users. Find at least three representative users from your target population. None of your users should be enrolled in 6.813/6.831. All should be willing to participate voluntarily.

Prepare briefing and tasks. These may be the same ones that you used in paper prototyping, but you may need to improve them based on feedback from the paper prototyping. (You may, if you wish, also prepare a short demo of your interface that you can use to show your users the purpose of the system. The demo should be scripted, so that you do and say the same things for each user. It should use a concrete example task, but the example task should be sufficiently different from the test tasks to avoid bias. The demo option is offered because some interfaces are learned primarily by watching someone else use the interface. Think carefully about whether your interface is in this category before you decide to use a demo, because the demo will cost you information. Once you've demonstrated how to use a feature, you forfeit the chance to observe how the user would have learned to use it alone.)

Pilot test. Do a dry run of your briefing, demo, and tasks before you test your real users. You can use another member of the class for your pilot testing, if you wish.

Run the test. Conduct a formative evaluation with each user:

• Provide your briefing and (optionally) demo.
• Then provide the tasks one at a time, observe, and take notes.

One member of your group should be the facilitator of the test, and the rest should be observers. Watch and record critical incidents. We don't recommend that you videotape your users. However, if you want a record of the user test to supplement your notes, you may try using screen capture software, such as QuickTime Player, which is free, or Camtasia Studio.

Analyzing results. Collect the usability problems found by your user tests into a list. Assign each problem a severity rating (cosmetic, minor, major, catastrophic), and brainstorm possible solutions for the problems.

Google Doc Project Report. Under the section "GR6 User Testing" add the following sections:

• Design

  Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing).

• Implementation

  Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface.

• Evaluation

  Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population (but don't identify your users by name). Describe how the users were briefed and what tasks they performed; if you did a demo for them as part of your briefing, justify that decision. List the usability problems you found, and discuss how you might solve them.

• Reflection

  Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.

Self-assessment form. Complete your individual self-assessment forms by the deadline.

Note that there is no studio presentation of this work. As usual, be complete, deep, and concise in your report. Tidy up your entire report to make it a usable presentation of your term project. If your project changed direction or scope over the course of the semester, update earlier sections (such as the original Problem section you wrote for GR1) to reflect your final project.