6.813/6.831 User Interface Design & Implementation

Due at 11:59pm, Friday, April 22, 2016 by uploading submission to Stellar.
You must complete the self-assessment form before you hand in the assignment.

This problem set is for the graduate 6.831 class only.

This problem set is the second in series (RS1, RS2, RS3) about reproducing a controlled experiment from a published paper. In this problem set, you will run the experiment on some people and collect data.

Human Subjects Training (20%)

Before running users, you should take and pass the Human Subjects Training course offered by MIT's Committee on the Use of Humans as Experimental Subjects:

The course is an online course. When you register for the course, the curriculum you should choose is Social and Behavioral Research Investigators. The course material consists of about 15 web pages, each followed by a short quiz about what you've read. You must score at least 90% overall to pass, but you can repeat any module and retake its quiz if necessary. Once you have passed the course, you will get a certificate of completion.

We're requiring you to take this online course for two reasons. First, its content is relevant to this class and important to anyone doing user testing. Second, MIT requires all researchers involved in experiments with human subjects to pass this online course. So you will come out of this class already certified to do user studies in your graduate research.

If you had previously passed the training course, you don't need to take it again. Just hand in your certificate of completion or some other evidence that you passed. The training site includes a list of all people who have passed the course, so if you're not sure whether you've done it, you can scan that list for your name and date of completion.

Print your completion report by clicking Print from the CITI Main Menu page, and save it to a PDF file. Include this PDF file when you submit on Stellar. If you don't have a printer driver that can generate PDF, there is free software for printing documents out to PDF.

Data Collection (70%)

Conduct a within-subjects controlled experiment of either ephemeral adaptation or bubble cursors, using the study infrastructure you built for RS1. (If you were unable to get your RS1 implementation working, contact the teaching staff.)

Your experiment should have 12 participants. Recruit your 6.813/6.831 classmates as much as possible (see below), but you can also recruit participants outside of the class.

Follow the protocol of:

  • Experiment 2 if you're replicating the experiment from the Ephemeral Adaptation paper
  • Experiment 2 if you're replicating the experiment from the Bubble Cursor paper.
Note that you only have two conditions (one control and one experimental condition) where the experiments in the papers had three or more, so your sessions should be shorter. You can reduce the length of your experiments so that it can be completed in roughly 15 minutes or 500 mouse clicks. Be sure to randomly assign your participants to conditions (e.g. orderings of tasks).

Collect your raw data in a clearly labeled spreadsheet, showing conditions (values of independent variables) and results (measured dependent variables, i.e. time). Be sure to include users' subjective ratings of difficulty, satisfaction, efficiency, and frustration. You don't have to run statistical tests, generate graphs, or compute summary statistics for now -- RS3 will do that.

User Study Participation (10%)

You are required to be a participant in three of your classmates' RS2 user studies. You will have opportunities to do this during class worktime, and a sign-up sheet will be available on Stellar to help you connect at other times.

What to Hand In

You must complete the self-assessment form before you hand in the assignment.

Submit the following as PDF files on Stellar, and then complete the self-assesssment form:

  • your completion report for human subjects training
  • a well-formatted and clearly labeled PDF version of your spreadsheet's raw data.
  • Enter the names of the three people for whom you acted as an RS2 participant here . (Note this is not a list of your own study's participants! You should deidentify your own participants in your data.)
  • a list of the people you discussed this assignment with, or explicitly say that you discussed the assignment with nobody. (you can put this at the top of your spreadsheet),. This is an individual assignment, so be aware of the course's collaboration policy.

Grading

This assignment will be judged on whether you did all the parts and presented them in a way that was easy to determine that you did them.