|
HomePage |
|
|
|
|
TEAL Assessment and Evaluation
The TEAL Project has had a robust assessment and evaluation effort underway since its inception. This effort is led by Professor Judy Yehudit Dori (http://caes.mit.edu/people/dori.html), a faculty member in the Department of Education in Technology and Science at the Technion. Professor Dori is an internationally-known educator whose expertise is the assessment of learning strategies in science and technology education. We use a variety of assessment techniques, including the traditional in-class exams, focus groups, questionnaires (in addition to MIT's CEG questionnaire), and pre- and post-testing. We concentrate here on the results of the pre- and post-testing. Our pre- and post-tests consists of 20 multiple choice questions covering basic concepts in electromagnetism. Some of these questions are taken from standardized tests that have been developed and used at other institutions, and some of these questions were developed at MIT.
The figure shows the results of the pre- and post-testing for spring 2003 8.02 (Dori and Belcher, 2004). The results are given for three categories of student scores: High, Intermediate, and Low. This separation allows us to gauge the effectiveness of instruction across the range of student backgrounds; the separation is made using the student's score on the pre-test (the dividing lines are: greater than 45/100; between 30 and 44/100; and less than 30/100). The difference between the pre- and post-scores is a measure of the effectiveness of instruction.
The table shows these results in the standard form for assessment studies using the normalized gain <g> (Hake, 1998)
In calculating <g> we are normalizing the student's improvement in his or her score from the pre- to the post-test to the maximum improvement possible. We also show in the table the pre- and post-test results for the TEAL prototype taught in fall 2001 and for a control group that consisted of 121 volunteers from the spring 2002 course, which was taught in the traditional lecture/recitation format (Dori and Belcher, 2003).
The table demonstrates that in spring 2003 our normalized learning gains are the highest we have achieved in the TEAL format, and are broadly spread across all levels of student background. The achieved <g> = 0.52 for all 514 students is comparable with <<g>> = 0.48 of 48 interactive engagement mechanics courses surveyed by Hake (1998), where the double angle brackets mean an average of the averages. They are better than the results in fall 2001 TEAL, both absolutely and in the spread of the learning gains across student background. Moreover, the learning gains in TEAL spring 2003 by this measure are about twice those in the traditional lecture/recitation control group of spring 2002. The fact that interactive-engagement teaching methods produce about twice the average normalized learning gains when compared to traditional instruction replicates the results of many studies obtained at other universities, including Harvard.
Any comparison between the spring 2002 control group and the spring 2003 TEAL group has a number of limitations. One might be concerned that the TEAL spring 2003 instructors "taught to the post-test," but in spring 2003 the six section leaders had almost no knowledge of the content of the pre- and post-tests, so "teaching to the test" was not a significant factor. There are other limitations which stem from the fact that not all the variables in the TEAL groups and the control group were identical: unlike the TEAL students, who responded to both conceptual and analytical problems as part of their 8.02 course work, the control group students only solved analytical problems in their weekly assignments and on the course exams; the conceptual pre- and post-tests administered to the TEAL group students were mandatory, whereas the control group students volunteered to take the pre- and post-tests and were compensated for their time; students in the TEAL groups were encouraged to attend classes because they got credit for doing so, while the control group had no such encouragement; the TEAL students consisted of the entire class population, while the volunteers in the control group accounted for about 20% of their classes (however, the average final grade of the volunteers in the control group traditional course was 66/100, higher than the average score of 59/100 for the entire control group class, so the volunteers were not unrepresentative of the abilities of the entire class); the control group is from spring 2002, when the Pass/No Record system was still in effect; in spring 2003, the grading system was ABC/No Record, which undoubtedly increased student motivation to do well in the course.
|
HomePage |
|
|
|
|