FNL
HomePage
Editorial Board
E-mail FNL
FNL Archives
MIT HomePage

OpenCourseWare Update

Measuring Long-Term Success
Evaluation of MIT OCW Depends on Articulation of Clear Goals

Kyung Han

When MIT OpenCourseWare was first announced in April 2001, the BBC came calling, President Vest and Hal Abelson suddenly became very familiar names to the readers of The New York Times, and the MIT News Office was deluged with e-mails from people all over the globe, excited about this bold educational initiative.

The announcement was a success.

When the first sample of courses opened to the public on September 30, 2002, MIT OCW received more than 13 million hits, the site did not crash, and the Institute once again garnered positive media coverage.

The MIT OCW pilot site was a success.

The sites developed for the first 50 MIT subjects in the pilot proved the concept, but measuring long-term success will not be so simple. There is no revenue model, so we will not judge success by profits. Visitors to the site cannot register for courses, so we can't benchmark the number of degrees or certificates we hand out. And it is a wholly new way of thinking about Web-enabled educational publishing, so we can't compare ourselves to similar initiatives. The ultimate evaluation of success will be whether we accomplish our dual mission: success in what we do – provide free, searchable, coherent access to all MIT course materials for educators, students, and individual learners around the world – and success in how we do it – creation of an efficient, standards-based model that adds value for the MIT faculty, and that other universities may emulate to publish their own course materials.

"Why would we do this?" President Charles M. Vest asked in April 2001. "Because we see it as part of our mission: to help to raise the quality of higher education in every corner of the globe. This program is based on the twin values of opportunity and openness. These are values that have made our universities and our nation strong. They are values that will keep our world safe and strong."

Although the vision for MIT OCW as stated by President Vest includes a major potential impact on an extremely broad audience, we must develop measurable goals. Just as importantly, these goals must be directly linked to MIT's mission as an institution. Now that the initial goals framework of what and how has been established, we are developing a set of sub-goals that support that framework and MIT's mission, and we are finding metrics that will measure our success.

Evaluation will also occur in stages: Short-term, we are focused on the publication of 500 courses in September, but long-term, we must plan for a way to measure MIT OCW's growth from the publication of 500 subjects, to the publication of nearly 2,000, virtually all of the subjects taught by MIT's faculty.

Evaluation of the what will address three areas:

Measured success against three metrics will inform and influence the strategic direction of the project, and will provide input and data for incremental refinements to MIT OCW materials and services.

Measuring the how of the project depends on development of effective strategies in five areas:

The initial version of the plan has been analyzed by a team of external experts, including John Seely Brown, the chief scientist of the Xerox Corporation; James Spohrer, a distinguished scientist at Apple Computer best known for his work in the areas of authoring tools and online learning communities; and Albert Koers, the executive director of the InterAcademy Council. Here at MIT, the goals and evaluation plan will also be scrutinized by the MIT OCW Faculty Advisory Committee, chaired by MIT Professor Steve Lerman.

As the evaluation and measurement of MIT OCW continue to evolve, and we begin to develop hard data that demonstrates the impact of this initiative, we will share them with the MIT faculty in this space.

If you would like to participate in MIT OCW, please contact Jon Paul Potts, the MIT OCW communications manager, at jpotts@mit.edu or 2-3621.

FNL
HomePage
Editorial Board
E-mail FNL
FNL Archives
MIT HomePage