PS5: Heuristic Evaluation

Participate in 3 user studies being done by students in the graduate version of the class. Go to:

study assignment form

to get links to 3 studies, which you can do online. Expect to spend about 15 minutes per user study. Some may have "ending conditions" (the experiment will "end" after a certain period of time) whereas others do not (spend ~15 minutes with the interface in that case). Once you've done them, check them off on the form and submit the form.

If the study doesn't start with instructions telling you what to do, then use these default instructions, depending on what the study is about:

• Sudoku puzzle: In this experiment, you will solve mini Sudoku puzzles. The rules of Sudoku are to fill in all the digits so that no duplicates appear in any row, column or bold-bordered square. Practice on puzzle #1. Then starting with puzzle #2, solve as many puzzles as quickly and correctly as you can.
• Calendar events: Use this interface to create 10 or more different events from your actual schedule this week, working as quickly and as accurately as you can.
• Menu selection: Each screen shows a prompt at the top with the name of the menu to choose and the item to choose from that menu. Choose items as quickly and accurately as you can.
• Target selection: Each screen will show a filled-in circle amidst a group of other empty circles. Click on the filled-in circle as quickly and accurately as you can.
• Image classification: You will be looking for images that match a certain category (e.g. images that contain a dog). Press the spacebar whenever you see an image that belongs in the category.

If you are assigned a link that doesn't seem to work at all -- you can't see the web page or even do the first task -- then please email 6.813-staff@mit.edu with the URL, an explanation of what you're seeing, and what operating system and browser you're using. The course staff will put you in touch with the study creator.

For this portion of the assignment, you will do heuristic evaluation on a computer prototype developed by your classmates. Go to:

heuristic evaluation assignment form

to receive the link to the Google Doc for the group you will evaluate. You should read the background information for the project you were assigned to (particularly the problem statement and target users), and look at the GR4 section for instructions for running the prototype and information about the project. Note which parts of the prototype are still shallow in implementation (which should be mentioned in the group's GR4 section), so that you're not surprised when those features fail to work as expected. This is not an anonymous evaluation, so feel free to contact the project group directly if you need more information than they provide.

As soon as you receive your prototype assignment, try to run the prototype. You don't have to do your heuristic evaluation right away, but poke around a bit and make sure the prototype appears to work for you. If you have any difficulty getting a prototype to run , send email to the people who created it, and CC 6.813-staff@mit.edu on your email. Do this trial run by Monday, April 23. We need to get logistical problems out of the way as early as possible.

Follow the heuristic evaluation procedure we discussed in class to evaluate the interface carefully. Make a numbered list of usability problems and successes you find. For each problem or positive comment, you should:

• describe the problem or positive feature, using screenshots
• identify the relevant usability heuristics (from the guidelines we've discussed in class)
• estimate its severity (for problems, use cosmetic, minor, major, or catastrophic; for positive comments, just say good)

You aren't required to recommend solutions for the problems, but any ideas you have would no doubt be appreciated.

Be thorough. Cover every major part of the prototype with your analysis. You should have at least 15 useful comments (positive or negative) about the interface that you evaluate.

Be thoughtful and analytical. Try to make use of all the usability heuristics.

Write your report in a readable style. The usability of your report to its recipients will matter in your grade. In particular, don't bury the problems you found in reams of free-flowing prose. Where possible, include screenshots to illustrate the problems you found. In general, make your report easy to read and understand.

1. Complete the self-assessment form (link at the top of this page) before you hand in the assignment.

2. Submit the study participation form (link above) to record whose user studies you participated in.

3. Your heuristic evaluation report must be in PDF format. Submit in two places:
   • Gradebook as in other assignments (link at the top of this page).
   • Email to all the members of the group, using their MIT usernames shown in their Google Doc.