Modeling Human Behavior in a 2-Choice Task

Purpose

The purpose of this project was to create models to predict human behavior in an experimental decision-making task. We designed and performed an experiment in which subjects had to choose between two buttons to receive a reward over several trials. After examining the data, our individual groups made models based on various factors that we thought were important in decision-making. The models were trained and tested on the data that we collected.

Experiment Description

Our experiment was based on a study by Egelman, Person, and Montague (1998), who attempted to model the role of dopamine in decision-making. Subjects were given the choice between two buttons. Each button gave some reward, however one button gave a greater reward than the other. The reward was represented on the computer screen by two bars (Figure 1). The red bar indicated the reward, in fractions of Hershey’s kisses, from the current choice and the blue bar indicated the cumulative reward over all of the trials. In our preliminary study, the subjects received candy at the end of each task. However, we found that subjects in the preliminary experiment were not very motivated to optimize their reward. In our second experiment, we changed the interface such that the game stopped each time the cumulative reward bar reached a whole increment of a Hershey’s kiss. The game instructed the subject to obtain their reward from the experimenter. The subject was then given their reward and allowed to either eat it immediately or set it aside and continue with the game.

Prior to beginning the experiment, the subjects were given the following instructions.

Instructions for the experiment

Each subjects participated in three different tasks, each of which consisted of 240 choices (or trials). The rewards for each choice were calculated based on the reward curves in Figure 2 and the previous 40 button presses.

Figure 1

Figure 2

Results and Collected Data

Data collection was performed in two phases. The first phase corresponded to the preliminary study, in which motivation was found to be a problem. The second phase corresponded to the second experiment, in which the interface was changed. The matlab data files corresponding to the subject choices and rewards are included in the corresponding .mat files. In each file, the following variables are present:

Var Name	Dimensions	Description
c1	num_trials x num_subjects	choices corresponding to subjects that performed task 1 in forward order
c1r	""	choices corresponding to subjects that performed task 1 in reverse order
c2	""	choices corresponding to subjects that performed task 2 in forward order
...	...	...
r1	""	rewards vectors corresponding to choices from c1
r1r	""	rewards vectors corresponding to choices from c1r
...	...	...

In the first phase, fifteen subjects were tested, with eight subjects performing the tasks in forward order (1, 2, 3), and seven subjects performing the tasks in reverse order (3,2,1).

phase1_data.mat

In the second phase, nine subjects were tested, with six subjects performing the tasks in forward order (1, 2, 3), and three subjects performing the tasks in reverse order (3,2,1).

phase2_data.mat

Modeling Approaches

Each pair of group members created a model to predict how the subjects would play the game.

Group Members	Project Title
Amy and Helen	Modeling of behavior based on reward history and personal risk factor
Christine and Alex	Statistical modeling of user switching behavior based on reward histories
Shaun and Jon	Transition Differential Summation Model

Presentations

Here are the Powerpoint slides for our class presentations:

Amy and Helen
Shaun and Jon
Christine and Alex

Matlab Code

Code for running the experiment (requires Matlab v6.5)

behavior3.m

behavior3.fig