RS3: Analysis
Due at 10:00PM, Friday, May 12th, 2017
by uploading submission to Stellar.
You must complete the self-assessment
form before you hand in the assignment.
This problem set is only for the doctoral 6.831D version of the class.
This problem set is the last in a series (RS1, RS2, RS3) about reproducing a controlled experiment from a published paper. In this problem set, you will analyze your results and write them up.
Here are useful video tutorials picked by the course staff:
Plots in R:
AVOVA in R:
AVOVA review:
- Calculating SST (total sum of squares)
- Calculating SSW and SSB (total sum of squares within and between)
- Hypothesis Test with F-Statistic
Experiment Analysis (50%)
Analyze your data. Was there a significant difference in performance among your conditions? To answer this question, you should first graph your data in an informative way, including error bars. Next, perform a test for statistical significance. Use the paper you're replicating to guide you in how to display and analyze your results. Note, however, that you did fewer conditions than the paper did, so the details of your test (the number of factors in the ANOVA, for example) may differ from theirs. Finally, discuss how the users' subjective feedback compares to the collected data and statistical analysis.
Report (50%)
Write up your results in a brief report, at most 3 pages long. Your report should have the following parts:
- Method
- Apparatus: the physical equipment you used
- Conditions: the independent variables
- Measures: the dependent variables you measured
- Procedure: a description of what participants did in your study
- Participants: how you recruited your participants, their general demographics (age, gender, occupation), and any other factors important to your study (e.g. handedness). Do not include personally-identifying information about your participants.
- Results: graphs and statistical tests and discussion of your results
Follow the model of the paper that you're replicating; pay close attention to how it describes the study and how it presents its results. Be sure to clearly state where your replicated study differs from the original, both in experiment design and in outcomes. Also, if you had to drop any participants or discard some data, state that explicitly and provide reasons.
If need help with your writing, not just with RS3 but with any professional communication, the EECS Communication Lab can help. Sign up for an appointment to get one-on-one feedback from a communication lab advisor, who is a grad student or post-doc trained in communication skills. Whether you have a draft or haven't yet written down a word, they can provide guidance; the earlier in your process, the better. You can make an appointment at the EECS Communication Lab website.
What to Hand In
Submit your report as a PDF file on Stellar, using the link at the top of this handout. You must also complete the self-assessment form linked at the top before you hand in the assignment.