The MIT Physics Department's Experience with edX

John Belcher

I give a brief summary of the Department’s planning for and experience with 8.02x, an online physics course offered last spring through edX. [This article is based on an article of the same name to appear in the fall 2013 issue of physics@mit.edu, the annual journal for members and friends of the MIT physics community.]

The MIT Physics Department has a long history of innovation in physics education, beginning with the formation in 1956 of the Physical Science Study Committee led by Jerrold Zacharias and Francis Friedman. More recent highlights include the enormous, world-wide impact of the Web physics lectures of Professor Walter Lewin beginning in the 1990s and continuing to the present, and the implementation in the early 2000s of the widely emulated interactive physics classroom instruction model for residential education, TEAL [Technology Enhanced Active Learning]. Given the Department’s current prominence in the GIR educational experience, and its long history in physics education, Professor Edmund Bertschinger, the then Physics Department Head, positioned the Department to become involved in edXat an early stage in the last academic year, both to gain experience in using the online platform and also to influence the capabilities of that platform by being an “early adopter.”

To that end, Bertschinger appointed an ad hoc committee, the PhysicsX Planning Group (PxPG),to create, monitor, and review the development and offerings of the first few departmental subjects on edX, and to advise him on the scheduling of subjects to go online. The online course 8.02x was approved in April 2012, before PxPG was formally convened. That course began online on February 18, 2013, and ended on June 17, 2013.

The decision to begin with 8.02x was made because the Department has available a wealth of digital resources for electromagnetism, most notably Lewin’s lectures from his residential course given in spring 2002. There was also an electromagnetism textbook and a number of electromagnetism simulations and visualizations created in the process of the curriculum development for 8.02 TEAL (this work was funded by the d’Arbeloff Fund for Excellence in Education and iCampus).

The next course to go online, 8.01x, was approved by PxPG in December of 2012.This course is scheduled to go online on September 9, 2013, with an end date of January 13, 2014.One additional course during the summer, Professor David Pritchard's Mechanics ReView, was approved in March of 2013, and began on June 1, 2013. Although not based on 8.01, this course is part of the Physics Department's effort to learn how to use the edX platform for improving teaching and learning.

The Format of 8.02x

The recorded lectures of Professor Walter Lewin's courses, including 8.01 (Classical Mechanics), 8.02 (Electricity and Magnetism), and 8.03 (Vibrations and Waves), have been visited more than 8 million times on MIT OpenCourseWare (OCW), and more than 11.4 million times on YouTube. The first lecture for 8.01, recorded in fall 1999, has been viewed more than 1.2 million times on YouTube. Translations of Professor Lewin's courses into Chinese, Spanish, Portuguese, Korean, Turkish, and Thai have been accessed by hundreds of thousands of learners.

Given Lewin’s immense reputation online, the Department decided to build the online offering of 8.02x around the 36 recordings of Lewin’s spring 2002 lectures, complete and in sequence. Because the OCW videos of these lectures were put online in 2003, at fairly low resolution, the decision was made to go back to the original tapes of Lewin’s lectures and redigitize them, thus producing much higher resolution videos for 8.02x compared to versions previously online. The only changes in his original lectures were edits by Professor Lewin to include three to approximately 10 “gaps” for short questions during a given lecture. Students viewing these lectures online in 8.02x had to answer these gap questions before they could proceed with the next lecture segment. This was done to provide more engagement and a better learning experience for the online students. This feature did not exist in the edX platform before the Department requested it, and it is an example of how interacting with the edXsystem early on helped the Department to mold its capabilities. In addition to the 2002 lectures, 8.02x used the many “help sessions” that Professor Lewin has recorded over the years for the weekly problems sets assigned in the residential version of 8.02. Also, many of the TEAL electromagnetic simulations were ported over to the edX platform for use in 8.02x, where they run in javascript on any HTML5 compliant browser, making the simulations widely accessible. The course also featured an online discussion board where the students could ask questions of the course staff, including Professor Lewin.

The structure of the online course was designed much like the residential course. There was a problem set due every three lectures, with three exams over the course of the term and a final exam.

To discourage dishonesty, the problem set and exams had problem parameters that were randomized between students – that is, different students received a different set of input parameters with a correspondingly different set of correct answers. To accommodate the various schedules of the online students and because of the limitations of the online format, the students had three days to complete the exams.

Thus, even though the exams were representative of an MIT level two-hour exam, the mean score on these exams were very high by MIT standards, with typically a third of the students getting a perfect score on the exams. Not only did the 8.02x students have three days to complete the exams, they also had access to any reference or textbook material they could find, including those on the Internet. This explains the high number of perfect scores in 8.02x compared to MIT students taking a limited-time two-hour exam of similar difficulty. We give the distribution of scores on the first 8.02x exam in Figure 1.

Figure 1. Distribution of scores
on the first 8.02x exam.
(click on image to enlarge)

Back to top

The Demographics for 8.02x

A total of 43,758 people registered for 8.02x through the end of the course. The academic background (the highest level of education attained) of registered students who chose to reveal that information is given in Table 1. The distribution varies somewhat if we look only at the students taking the exams, but not qualitatively. Not surprisingly, using data not shown in Table 1, the more extensive the academic background attained by a given student, the higher the grade that student scored on the exams, on average.

Table 1. Academic background of select registered students.

Of this large registered number, only 5,241 attempted the first problem set. The first one-hour exam was taken by 3,490 students, and the second exam by 2,459 students. A total of 1,715 students completed the course and received a certificate signed by Professor Lewin attesting to their having passed the course.

The distribution of ages of the enrolled and active students in 8.02x is shown in Figure 2.These students came from all over the world, with the largest percentage from the United States (19.0%), followed by India (17.2%), Spain (7.8%), the United Kingdom (4.5%), the Russian Federation (4.0%), and Greece (3.1%).

Figure2. Distribution of ages of enrolled and active students in 8.02x.“Active” means active just after the second 8.02x exam.
(click on image to enlarge)

Reception of 8.02x

Overall, it is clear that 8.02x was well received by most of the students taking it. For example, in an effort to achieve personal rapport with the students, Professor Lewin broadcast a short weekly video to encourage the students, interwoven with practical advice about how to do well (or better) in the course. The reaction to these videos was very positive, as witness the following quote from an 8.02x student:

Believe it or not, I so eagerly wait for Prof. Walter Lewin's video message each week. And once it is uploaded I watch it at least 4-5 times. He is such an awesome gentleman... Hats off to you Sir!!

The satisfaction of students with the course is also evident from the general tenor of the discussion board comments. For example:

Thank you for making this amazing class available for free. I took 2 classes on line before….Not only [is] Professor Lewin amazing, of course, but the structure of the class on line is outstanding [including the] mixture of videos, quizzes, simulations HW, textbook. The electronics class I took was not only expensive but of very poor quality. This class takes advantage of the best of internet/java/... I hope it stays free. I hope for more classes like Physics 8.03 and 8.01. Bravo and thank you again.

These anecdotal opinions were verified by an online questionnaire at the end of the course. The question “How likely are you to recommend 8.02x to a friend?” scored a 9.43 on a scale from 0-10, with 10 being extremely likely. When students were asked how much they agree that the TEAL visualizations helped to achieve course goals/learn, the results were 4.5 on a scale of 0-5, with 5 being "strongly agree," 1 "strongly disagree,” and 0 being "not used."

Back to top

Lessons Learned

Overall this experience has been a success in that the Department has succeeded in offering the course online with few technical problems. In doing this, we gave 1,715 students around the world an opportunity to take Professor Walter Lewin’s spring 2002 8.02 course in depth, and to receive a certificate from MITx. Moreover, we have facilitated the interaction of these students with Professor Lewin, with the course staff, and with each other. In doing this, the Department has gained invaluable experience with the edX platform and, by being an early adopter, helped shape the capabilities of that platform. The Department now has experience with an edX course, including how to code problem sets and exam problems, the quality control needed before such problems go “live”, how to moderate discussion boards effectively, how to motivate students, and many other aspects of running a large online course. The Department also now has the knowledge necessary to use features of the edXonline courses to improve our residential offerings in physics at MIT, which is a major goal of the MIT Office of Digital Learning, and of our Department.

What is less clear is how successful this course was in terms of learning outcomes. The distribution of grades on the first exam, as given in Figure 1, was unexpected, but obvious with hindsight. We had many very serious students taking the course, for no formal credit other than a certificate of completion, and they had three days in which to complete the test. It is not surprising that one-third of these dedicated students put in the additional hours necessary to get a perfect score on the exam, and that many did very well.

But this leaves us with little information of how to rank the students in terms of ability, other than in a very coarse manner. In a residential course, we get a much more detailed impression of the relative abilities of the students.

If we continue to obtain distributions such as given in Figure 1, varying the pedagogy of the online course will not tell us much about the most effective pedagogy, because we will generally end up with this kind of distribution, which tells us very little.

In the future, the edX platform may be able to require students to take the exam for a set length of time in a continuous fashion, sometime within a three-day window, but it does not have that capability now. Even if such a feature is instituted in the future, there are many obvious ways to defeat the system (by registering twice under different names, for example). The most straightforward way to get back to a grade distribution comparable to residential courses is to have a proctored limited time exam, and edX is exploring that option.

Another lesson we learned from this experience is that putting this kind of course online is no easy task. The people who worked on this effort are acknowledged at the end of this article. This is a group of talented people who dedicated many hours of their time to 8.02x. We are now past many of the start-up costs of mounting this kind of effort but, even so, refreshing 8.02x and offering it again, complete with an adequately staffed discussion board, is an expensive proposition. For example, putting a new homework problem online requires the problem statement, the coding to put that statement online, extensive testing of that code before it appears online, and a system to correct the mistakes that still leak through after it has gone online. Although we could perhaps simply repeat the problem sets from one online offering to the next, we would most certainly have to make new exams for each offering. Otherwise these exams and many sample solutions will be available online from previous offerings of the course, and the scores on those repeat exams in the next iteration of the course would be meaningless. For these reasons, offering edXcourses on a regular basis would require a substantial augmentation of the teaching personnel of the Department.

In conclusion, we have learned much in this process, but this is just the first of many steps in a long journey. The next issue that the Department plans to address is how to bring the capabilities of the edX platform back into our residential program, particularly in the freshmen subjects. A “flipped” class is one where lecture is presented online outside of class and hands-on-work (discussion, homework, experiments) is done in class. Our present way of teaching the majority of MIT freshmen, TEAL, is half-flipped already, in that 50% of the class time is devoted to group discussion, group problems, and hands-on experiments. We plan to experiment with completely flipping 8.02 TEAL for two to three weeks in the coming academic year, using the capabilities of the edX platform to deliver the online content. In doing this, we plan extensive assessment to compare the half-flipped and completely flipped parts of 8.02 residential, including gathering both student and faculty opinion and evaluating student learning gains. Since we have much more interaction with and knowledge of the student population on campus, this use of the edX platform in residential may also offer us useful insights as to what to incorporate into the “x” courses in future incarnations.

There is much that remains to be done.

Acknowledgments

Participants in putting the course on line were Professor Lewin, Dr. Riccardo Abbate, Dr. Saif Rayyan, Dr. George Stephans, Professor Isaac Chuang, Dr. Peter Dourmashkin, graduate teaching assistants Peter Montag, Anthony Zhu, Agata Wisniowska, and UROP students Maita Esteban, Mario Martenez, Zixi (Zeo) Liu, Marie Rice, Thomas Villalon, Troy Welton, and Justin White. Professor John Belcher was the faculty project manager. Dr. Teppo Jouttenus was the edX project manager.

	Back to top
	Send your comments