8.02 TEAL+x: Students Say "Yes" to MITx in 8.02 TEAL

Saif Rayyan, John Belcher

In spring of 2014, we ran an experiment to explore the use of the MITx platform in on-campus 8.02. The platform was developed by edX for presenting MOOCs (Massive Open Online Courses), and has the potential to be useful in MIT's residential courses. To this end MIT's Office of Digital Learning (ODL) has had its MITx group implement the edX platform on campus, and we have used it to enhance student experience in 8.02, MIT's physics course on electricity and magnetism taken by over 800 freshmen.

We concentrated on providing automated feedback to activities done outside class: mainly pre-class preparations, and homework. Students received the platform very favorably, and an overwhelming 95% answered “Yes” to the question of whether we should continue to use the MITx platform in physics courses. In the future, analyzing the voluminous click stream logs will enable us to better evaluate the efficacy of the online component.

TEAL and the “Flipped” Classroom

 “Do no harm”: when we set out to explore the potential of using the MITx platform to improve our teaching, our first priority was to avoid impacting negatively what we think already works well. 8.02 is currently taught using the TEAL (Technology Enabled Active Learning) classrooms. Class activities are motivated by a large volume of education research showing that effective use of peer instruction, group problem solving, and interaction between staff and students in class all lead to better learning. These elements exist in one way or another in what people call the “flipped” classroom, and, in a sense, we have already done our first attempt to flip the classroom when we moved from lecture-recitation format to TEAL. We envisioned that the MITx platform could help us increase the time spent in TEAL on interactive activities, but, more importantly, we recognized the chance to improve student learning from activities outside class. We designed the MITx component to provide immediate feedback while students prepare for class and while they do their homework.

The x in TEAL+x

Pre-class
Getting students to prepare for class has always been a challenge. Previously, pre-class assignments in TEAL were a combination of assigned readings from the textbook and the submission of handwritten answers to one or two open-ended questions. The papers were collected at the beginning of each class, and graded by undergraduate TAs. That system did not work: We did not believe our students read the material, and they did not receive immediate feedback to their answers.

For 8.02 TEAL+x, pre-class assignments were transformed using the MITx platform to be due right before class three times a week. The assignments started with a “reading summary” with links to the appropriate chapters in the textbook (all embedded in MITx, to the relevant page). The reading was followed by a few questions that targeted basic applications of the concepts discussed in the reading summary. These questions were automatically graded by the platform, and credit was based on effort: Students were able to look at feedback explaining the answer after submitting the first attempt; hence they could always get full credit. The goal was to allow students to test their understanding of the reading material by attempting these questions on their own first, and then providing them with immediate feedback. Pre-class assignments were worth 5% of the total grade.

Homework
In previous semesters, 8.02 TEAL homework was submitted on paper, and graded by TAs. Ideally, TAs graded two-to-three problems out of eight, and were expected to return graded assignments within a week, by which time students were working on the next assignment and were no longer focused on the previous assignment.

For 8.02 TEAL+x, we coded all homework problems into the MITx platform to enable students to check their intermediate and final answers. There they enter an answer and the system tells them if they are correct or not, with a green check mark if correct, or a red cross if incorrect. Students could see whether their answer was correct or not, but they were not given answers or solutions until after the due date.

Students were asked to submit answers to two of the homework problems online, and submit handwritten solutions to the remaining six problems. Students were offered the use of the online checker for the handwritten problems, but it was not required. We hoped the checker would encourage students to focus on the process of solving the problems rather than getting the final answer. Because we were not sure we could transform all problems into the MITx platform to be automatically checkable, we kept most of the process of submitting written homework unchanged. Homework was worth 10% of the total grade.

Supplementary material
The full set of lectures by Prof. Walter Lewin was included as a supplementary resource. TEAL simulations, video visualizations, and the textbook were all included in the platform – material that was available from the past offering of 8.02x on edX. We also linked to “Piazza,” an online forum for discussions where students can ask and answer questions.

Feedback from TLL and Student Committees

We assembled a team of instructors and students to plan and create the online course supplement on MITx. In the planning stages, we also consulted the Undergraduate Association Student Committee on Education (CoE), chaired by Anubhav Sinha ’14, and the Undergraduate Association Student Committee on MITx, chaired by Colin McDonnell ’16. We also consulted with Dr. Lori Breslow, Dr. Glenda Stump, and Dr. Jennifer DeBoer from the Teaching and Learning Lab (TLL), who attended many of our group meetings for designing 8.02 TEAL+x, and were an invaluable source of student perspective. We arranged to collect as much feedback from students as possible, including surveys and personal meetings with some students.

Back to top

Overwhelming student approval

In Week 11 of the course, we conducted an anonymous survey (replacing one of the pre-class assignments) to collect feedback from students on various online resources in MITx. Of the 800 students, we had 573 respondents. An amazing 95% of students answered “Yes” when asked if they thought 8.02 should continue to use MITx, and 92% thought that other physics courses could benefit from using the MITx platform. It was clear from the responses that students appreciated the value of immediate feedback, especially for the homework checkable answers.

The figure below shows the responses to the question “Indicate how helpful the following resources and activities are in learning the 8.02 material.” An overwhelming majority of the students (92%) rated the checkable answers on MITx for written homework problems to be “extremely helpful” (79%) or “very helpful” (13%). When asked to justify their answers, the majority of students raved about the value of automated feedback for checking homework answers on the MITx platform. It enabled them to know when they made a mistake and learn from it before submitting the homework. It also reduced their stress about the homework, and raised their self-confidence.

Student rating of the benefit of various online components of 8.02
(click on image to enlarge)

In contrast, the reaction to pre-class assignments was mixed. More than 20% of students indicated that they did not use the reading summaries or the readings from the textbook. Students split on the value of the reading material and the pre-class questions. The split also was evident in their comments, where some of them found the pre-class assignments very useful in preparing for class, and others found them difficult to complete in time and did not see the point of the exercises. In spite of the mixed reviews on the value of pre-class material, 85% thought that it is better to do the assignments on MITx rather than submitting them on paper. We were able to ask that question because most of the students in 8.02 took 8.01the previous semester, where the old paper format reading questions were used, so students were able to compare the two from direct experience.

While we believe that Prof. Lewin’s lectures are phenomenal, watching them would add an additional 2-3 hours per week for a student, which would explain why most of our students did not use them. Similarly, students did not report extensive use of the TEAL simulations, and most did not find them very useful. This can be attributed to the fact that these simulations were already used in class to build discussion on key concepts, and there was no clear plan for how to use them online.

There remains the question of how much students actually benefited from the platform. While we have a lot of anecdotal evidence (including the subjective impression of the faculty who taught 8.02) that the platform helped in learning, we need to look at the data logged by the system and analyze it to correlate with performance, self-efficacy, and learning habits.

Challenges

The Physics Department is building on the success of 8.02 TEAL+x by developing similar tools for 8.01, and 8.01 TEAL+x is running for the first time this semester (fall 2014). At this stage, a substantial effort by many people is required to get the entire content ready in time with minimal bugs, especially for the online checkable answers. As we continue to use the platform, we hope to build a library with enough validated problems to draw from year after year, reducing the effort needed to use the platform. With less time required to mount and manage the course, we will be able to focus more on using course data to improve the pedagogy and content for future use.

Acknowledgements

This work would not have been possible without the substantial efforts by Dr. Jolyon Bloomfield, Prof. Charles H. Holbrow, Dr. Peter Dourmashkin, Dr. George S. F. Stephans, Chester Chu, and many wonderful undergraduate TAs and UROPs. We also thank Dr. Lori Breslow and TLL for their continuous support in assessing our educational practices and helping with the instructional design. Elements of the course design were inspired by early experiments of using the MITx platform in other physics classes by Dr. George S. F. Stephans and Analia Barrantes. We thank ODL for providing the financial and technical support to make the project come to life.

	Back to top
	Send your comments