The Future of Learning Management at MIT

Dan Hastings and Hal Abelson

In 2009, the MIT Council on Educational Technology (MITCET) charged the Faculty Committee on Learning Management Systems (LMS) to provide strategic guidance on the future of LMS at MIT. MIT needs a more robust LMS with the flexibility and features necessary to support the evolving pedagogical and technological innovations in the classroom. The Committee, chaired by Eric Klopfer, collaborated with Information Services and Technology (IS&T) to gather community requirements, evaluate alternatives, and ultimately recommended the evaluation of the Blackboard Learning Management System as a possible alternative to the current Stellar platform.

Results of the Blackboard Evaluation

In spring 2011, IS&T conducted a limited evaluation of Blackboard 9.1. The evaluation included two components: a rigorous technical assessment of the Blackboard platform and extensive user testing. The technical analysis revealed several significant issues with respect to the Blackboard LMS, highlighted by shortcomings in the following areas:

• Supportability
• Maintainability
• Core functionality
• Extensibility and customizability
• Value-added functionality

IS&T concluded that, from a technical perspective, the systemic issues associated with supporting and maintaining Blackboard 9.1, coupled with limitations in core functionality and extensibility, render the product less than suitable to MIT’s needs.

At the same time, user testing showed that the majority of users found Blackboard more difficult to use and administer than Stellar. Testing involved 14 courses spanning six disciplines, representing the participation of 33 course administrators and over 600 MIT students. In order to mitigate the dissonance often associated with product migrations, the version of Blackboard 9.1 implemented at MIT was heavily customized to present a user interface and workflow logic closely paralleling that of Stellar.

In follow-up surveys, 90% of course administrators and 68% of students preferred Stellar over Blackboard. In considering the content management capabilities, 97% of administrators found the Blackboard Content Collection difficult to use and 87% bypassed it and managed content manually. Grading was also challenging; 62% of the administrators found the Blackboard gradebook tool difficult to configure and use. Finally, the Blackboard course support modules received mixed reviews, but approval rates were unimpressive overall.

Based on the consistently negative results of both the technical assessment and user testing, the Committee on LMS made a recommendation to MITCET to halt further experimentation with the Blackboard platform. The Committee also recommended a shift to the Modular Service Framework as the foundation for learning management at MIT. MITCET supported both recommendations.

The details of the analysis and user feedback are documented in the Blackboard 9.1 Experiment: Analysis and Recommendation Report.

Future Plans

In moving forward, IS&T has already shifted resources to the development of the Modular Service Framework as the new foundation for learning management at MIT. This approach will gradually replace existing Sakai 2-based Stellar functionality with a set of discrete, flexible Web services driven by a common data framework and based on a standardized set of Application Programming Interfaces (APIs).

With a focus on flexibility and integration, the Modular Service Framework is well positioned to meet MIT’s future LMS needs, including the technological evolution mandated by Digital MIT, as well as the emerging trends in curriculum development and online education. In this model, key functional components are represented by individual Web services that can be utilized as either standalone modules or as part of an integrated set of user tools. These Web services are driven by common core data sets, and share common standardized APIs. This aspect of the model eases integration and interoperability with community-developed and third party tools. Such an approach encourages community innovation while balancing individual customizability and extensibility with service standardization and the reduction of support overhead.
Development of the Modular Service Framework began in summer 2011. Current planning projects a 48-month delivery trajectory for the initial core components. This includes grading, attendance, calendaring, content and material management, forum integration, and blog/wiki integration. These represent existing functionality currently delivered within the Sakai 2-based Stellar framework. The first component, the Gradebook Module, has been made available to community evaluators as a beta 1 release; a full community release is planned for fall 2012.

Over the next several years, as existing core Stellar services are accounted for in the Modular Service Framework, focus will increasingly shift to the integration of value-added functionality satisfying specific unmet or emerging user needs. Such functionality will be identified and prioritized via a community requirement gathering process.

This new direction for Stellar will allow MIT to develop a state-of-the-art, Web-services enabled learning management system. In the future, the LMS will also provide integration points with the registration system, as well as advising and learning tools for both students and faculty. As a critical system serving the needs of a large percentage of the faculty, we realize that it is important to keep such functionality front and center as MIT continues to transition to a more digital future.

	Back to top
	Send your comments