AS2: A/B Testing Data Collection

Before running users, you should take and pass the Human Subjects Training course offered by MIT's Committee on the Use of Humans as Experimental Subjects:

• Human Subjects Training

The course is an online course. When you register for the course, the curriculum you should choose is Social and Behavioral Research Investigators. The course material consists of about 15 web pages, each followed by a short quiz about what you've read. You must score at least 90% overall to pass, but you can repeat any module and retake its quiz if necessary. Once you have passed the course, you will get a certificate of completion.

We're requiring you to take this online course for two reasons. First, its content is relevant to this class and important to anyone doing user testing. Second, MIT requires all researchers involved in experiments with human subjects to pass this online course. So you will come out of this class already certified to do user studies in your graduate research.

If you had previously passed the training course, you don't need to take it again. Just hand in your certificate of completion or some other evidence that you passed. The training site includes a list of all people who have passed the course, so if you're not sure whether you've done it, you can scan that list for your name and date of completion.

Print your completion report by clicking Print from the CITI Main Menu page, and save it to a PDF file. Include this PDF in the zip file you submit on Stellar.

Collect data on your B interface from AS1. (If you were unable to get your AS1 implementation working, contact the teaching staff.)

You should have at least 10 participants. Your 6.813/6.831 classmates (including you) will be asked to do some user testing as part of the PS5 assignment, but you can also recruit participants outside of the class.

The course staff will provide a mechanism for you to put your interface URL into an experimental pool that your classmates can draw from randomly for their user testing.

You don't need to collect data on the A interface. The course staff will collect that data by putting the A interface into the same random pool, and then share the collected A data with you so that you can compare it with your data in AS3.

Collect your raw data in a clearly labeled spreadsheet (but save in .csv format!). Add a column "User Number" which should be the user number (from 1 through 10 or more). Look at uid to determine unique users. You don't have to generate graphs or compute any summary statistics for now -- AS3 will do that.

Package your completed assignment as a zip file that contains:

• your completion report for human subjects training, as a PDF file
• your spreadsheet of data, as a CSV file (Google Spreadsheets can export CSV)

Submit your zip file on Gradebook (link at the top of this handout). You must also complete the self-assessment form (link at the top of this handout) before you hand in the assignment.

This assignment will be judged on whether you did all the parts and presented them in a way that was easy to determine that you did them.