OpenCourseWare Update

Beyond the Anecdotes

Steven R. Lerman

When MIT first announced OpenCourseWare in April 2001, it was just an idea – an informed leap of faith that it would be the right thing to do and that it would advance education. No university had ever done anything quite like it, and so we could only speculate on what the world might do with the educational materials we decided to make freely available.

Through the first year or so of the MIT OpenCourseWare (MIT OCW) operation, we were encouraged by the volume of laudatory feedback from users around the world. Now we are beginning to learn what OCW really means.

Just after its fall 2003 "official launch," publishing the first 506 courses for public access on the Web, MIT began a rigorous data collection process to find out who is accessing MIT OCW, why and how they use it, and what difference the initiative makes. The results of this first baseline evaluation confirm what we have heard anecdotally from more than 15,000 e-mails since the original pilot Web site of 32 courses was opened to the public in fall 2002: that educators, students, and self-learners around the world are using our course materials, and that, overwhelmingly, they find the materials useful in meeting their own learning and teaching goals.

The evaluation of MIT OCW integrates a variety of quantitative and qualitative methodologies and data sources: Web analytics are technical tools that capture site traffic data, geographic origin of visits, click patterns, and several other metrics. Most of these tools are in permanent operation, measuring activity on the MIT OCW Web site around the clock. During a two-week period in November 2003, MIT OCW asked about one in 40 visitors to participate in an online intercept survey , resulting in about 21,500 solicitations with a 5.7% response rate. These surveys asked visitors to profile themselves, indicate how they use MIT OCW materials, what they expected to get out of it, and their success in achieving their objectives. The MIT OCW conducted supplemental surveys and in-depth interviews with smaller samples to probe more deeply into the nature of their use.

Weekly visits to the OCW Web site . (click on image to enlarge)

Who is accessing MIT OCW? On average, MIT OCW clocks over 11,000 visits per day, with nearly a quarter-million unique visitors per month. About 45% of these visitors come from the United States and Canada. Outside North America, the top countries of origin are China, the United Kingdom, Germany, India, Brazil, and Japan. In total, about 52% of visitors identify themselves as "self-learners," 31% as "students" enrolled in a formal course of study, and 13% as "educators." Assuming our survey respondents are representative of the population of MIT OCW users at-large, approximately 90,000 educators visited the MIT OCW Web site between October 1 and November 30, 2003.

We view educators as a particularly important target audience, because it is through them that MIT course materials can touch the greatest number of people and have the most profound impact on education around the world. Interestingly, while the overall proportion of educator visits is 13%, the proportion of educators from outside North America is higher, about 17%. About 55% of educators profile themselves as teaching in four-year colleges or universities, another 18% in graduate or professional schools, and 8% in two-year or junior college programs. Nearly half of all educators using MIT OCW have less than five years teaching experience.

Back to top

Why and how are they using it? MIT OCW asked visitors their primary purpose in using MIT course materials. Again focusing on educators, about 57% answered that they use it for course or curriculum development, 33% to enhance their subject matter understanding or support their research, and 7% for student advising. The most frequently cited subject areas are electrical engineering and computer science, mathematics, and business/management. Elements of MIT course materials have been adopted or adapted for classroom use by 47% of educators, and another 41% are considering it.

Impact of OCW (click on image to enlarge)

What difference does it make? About 45% of educators indicate they were completely successful, and another 43% somewhat successful, in achieving their purpose when visiting MIT OCW. In the category of developing or planning a course, 74% of educators found MIT OCW materials useful or extremely useful. Over 97% of educators expressed satisfaction with the quality of the course materials published on the MIT OCW Web site, although anecdotal indications are that educators are more interested in the course structure and materials such as reference lists, exercises, and exams. About 76% agreed that MIT OCW will impact their future teaching practices, and nearly 99% believe that MIT OCW will have a positive impact on education overall.

These results are very gratifying, and demonstrate the profound impact we are having through the sharing of our course materials. Even in its early stages, MIT OCW is becoming an important educational resource. As of April 1, there are now 701 courses available, representing contributed materials from nearly half of our faculty. Moreover, while we continue publishing courses here at home, we have also begun talking with several other institutions that are interested in following in our footsteps, and some others are developing translations of some of the materials from MIT into their own languages. Our over-arching vision is that MIT's commitment to MIT OCW will be the start of a global movement in which faculty around the world create a body of educational materials that we all can draw upon in our teaching.

The full version (90 pages), and a summary version (14 pages), of the OpenCourseWare Program Evaluation Findings Report (March 2004) are available online at http://ocw.mit.edu/OcwWeb/Global/AboutOCW/evaluation.htm . If you would like to participate in MIT OCW, please contact Jon Paul Potts, the MIT OCW communications manager, at jpotts@mit.edu or 2-3621.

	Back to top
	Send your comments