Assessment: A Fundamental Responsibility

 

 

 

 

 

  

High stakes-

Important stakeholders

Improving teaching and learning is in the national interest. For individuals and for our nation, higher education opens doors to economic prosperity and civic engagement.  It provides the tools to make new discoveries and improve the quality of life.  It is understandable that students, parents and others want to be sure that the educations we provide are meeting the highest possible standards.  We share this same goal.  Like other stakeholders, we want to be sure that our assessment programs allow us to understand the extent to which our teaching and learning goals are being met.

 

 

 

 

 

 

 

 

 

  

 

 

Our Students,

Our Graduates

It may surprise some to know how much our colleges and universities have changed over their long histories and how much they continue to change, especially in reaching out to students from diverse backgrounds. For example, at COFHE schools, almost 30 percent of the students are people of color and some 47 percent receive financial aid.  The average award for students on financial aid is more than $28,000 per year.  Close to half of our students are involved in community service, and hundreds of thousands of our graduates make significant contributions to society through their work, their volunteer activities and their philanthropy.  For specific examples, please request a copy of “Take a Closer Look” from

cofhe-support@mit.edu

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Sharing responsibility for learning

Any assessment of student learning must take into account the responsibilities of students and of faculty.  Students bear responsibility for their educations in many ways, from choosing carefully what kind of institution to attend and selecting courses for which they have adequate preparation, to attending class and performing assignments to the best of their abilities.

 

Faculty bear responsibility for being knowledgeable in the subject matter, choosing appropriate pedagogies and  providing an environment conducive to learning by students with varying degrees of preparation, motivation and aptitude.

 

Effective teaching has moved from a model of the “sage on the stage” towards a “guide by the side” approach, emphasizing an active role for students combined with an active role for faculty as coaches and facilitators. Methods of assessment must include analysis of student performance coupled with analysis of the effectiveness of the faculty in supporting student achievement.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Accountability or Assessment?

It is important to distinguish between two related, but distinct, concepts: Accountability to institutional mission and assessment of teaching and learning.  “Accountability to institutional mission” refers to the idea that universities and other educational institutions should make data available by which external constituencies – including prospective students, parents, the government, and other stakeholders – can judge whether these institutions are performing their missions successfully.  This is an important commitment to the American social contract and should be engaged seriously by institutions of higher learning. “Assessment of teaching and learning” refers to the idea that institutions should continuously and rigorously review, and implement measures to enhance, the quality of instruction and student performance.

 

 

 

 

 

 

 

 

 

 

 

 

We come together as universities and colleges committed to educating undergraduates in the liberal arts and sciences.  We focus not just on what students know when they graduate, but on how their undergraduate experience prepares them for the future.  The most important measure of our success is not only the content our students learn, but even more importantly the habits of mind they develop (such as critical thinking, ability to evaluate alternative courses of action, resourcefulness and creativity) that will help them continue to learn and to cope with the challenges, changes and complexities inherent to a productive life.

 

We believe we have a fundamental responsibility to assess the effectiveness of our programs, while at the same time recognizing that the transformative impact that we aspire to have on our graduates is measured in decades, not in weeks or months.  This long-term horizon complicates our ability to assess the outcomes of the educations we provide and requires us to sustain our assessment process over an extended period of time.  It is far easier to measure a short-term achievement (for example, mastery of a particular skill or of a particular academic field at the time of graduation) than to measure a life-long attribute, such as intellectual curiosity or analytical rigor or a commitment to civic engagement. Standards of assessment that focus on what is easily measured run the risk of degrading the quality of the education that students receive (via pressure to teach to the test or only to learn for the test) and of diverting attention from the harder-to-measure but more important roles that our institutions play in fostering a thoughtful, engaged and broadly-educated citizenry.

 

As institutions we have always placed a high value on continually questioning whether we are doing our best.  As scholars we lead our professional lives according to the principle of peer review, subjecting our research and creative activities to rigorous assessment by specialists in our fields.  As teachers we welcome equally rigorous assessment of student learning.  We recognize the importance of examining systematically the quality of teaching and learning at our institutions, and we embrace opportunities to learn from each other.  We believe – as we demonstrate below – that we already engage in effective assessment in ways that are appropriate to our missions and that enhance the quality of the educations we provide to our students.  We also believe that we can always do better, and we have welcomed the opportunity to think together about how we might improve.

 

Diversity of Colleges and Missions

 

A core strength of American higher education is the diversity of institutions that contribute to the overall enterprise.  This diversity maximizes the opportunity for each student to find the specific environment that will best serve his or her interests and talents, thus optimizing the positive impact higher education can have on American society.  Many dimensions of this diversity are apparent: for example, whether a school is public or private; whether its student body measures in the hundreds or the thousands (or even tens of thousands); whether it attracts students regionally, nationally, or internationally; whether it is a research university granting graduate and professional degrees, a four-year college focusing primarily on undergraduate education, or a community college providing opportunities to students across a broad spectrum of ages and educational goals.  Other dimensions require closer scrutiny: for example, the level of selectivity in admissions; the specific demographic characteristics of the student body; or the sometimes idiosyncratic differences among institutions in their particular disciplinary strengths.  It is this diversity that makes American higher education distinctive; it also makes it the envy of the world, as evidenced by the very large numbers of foreign students seeking degrees at American colleges and universities.

 

While all of our institutions share a basic commitment to providing undergraduates with a liberal arts education emphasizing critical thinking, our differences mean that students will take different paths to achieve this end.  For example, larger institutions are more likely to have a substantial commitment to graduate and professional education, which provides students with access to cutting edge research programs, as well as to faculty in the professional schools.  Since liberal arts colleges tend to be smaller, their students are more likely to enjoy focused attention from the faculty to a degree not as readily found in a larger university setting.  Other differences, even among very similar institutions, reflect different strengths and priorities that are deeply woven into the fabric of each institution in a way that defines its characteristics.  At the same time, all of our institutions expect our undergraduates to acquire knowledge in a range of fields, develop a breadth of intellectual skills, and study at least one subject in depth.

 

The Value and Challenges of Assessment

 

Our colleges and universities take seriously their obligation to evaluate the extent of student learning, the effectiveness of teaching and the design of the curriculum.  Assessment is alive and well at our institutions.

 

The links below provide a small sample of the many examples of rigorous assessment that can be found across the country.  These examples illustrate several important ways in which assessment can be used to improve students’ educational experiences and outcomes.

 

Assessment can support decisions about the undergraduate curriculum.  As described in the linked example, one university has set aside funds that the Dean of Undergraduate Education can use to support new approaches to teaching, with some of the experiments involving individual faculty members and others cutting across several academic departments.  All of the projects are expected to have a plan for testing whether the innovation improved teaching. Another university has assessed whether changes made in the curriculum, combined with the availability of writing tutors, have been successful in promoting a “culture of writing.”  Yet another college has studied whether changes in its curriculum led to a larger number of students taking more advanced, literature-level courses in foreign languages.

 

Assessment can help faculty regularly revisit and update courses. One university is using student evaluations not only to provide feedback to faculty about the quality of instruction as is typically done, but also to assess the effectiveness of the course’s contribution to the development of higher-order skills. This assessment is then used to modify the course for the next time it is offered.

 

Assessment can provide information about long-term educational effects.   One college asked alumnae who had graduated ten years earlier to identify the areas of intellectual development that had been most important to them after college.  The areas they identified – the ability to write effectively, to communicate orally, to acquire new skills on one’s own, among others – became the priorities for the college’s continuing efforts to strengthen its curriculum.

 

Assessment can help individual faculty members constantly evaluate whether students are learning.  In every college and university, assignments, papers, projects, and exams are crafted to determine how well students have mastered the material and techniques of a given subject.  Students learn through constructive criticism, and through grades that demonstrate to the student, and to external audiences, the degree to which the student has mastered the skills and knowledge the course is intended to impart.

 

 

Assessment can evaluate the development of higher-order skills.  One college developed a program to examine not just whether students are gaining mastery of a body of knowledge, but whether they are developing the ability to conduct original research and critical analyses.  This college requires rising sophomores to submit a portfolio of three to five papers from at least two areas of the curriculum.  The students are expected to demonstrate that they have mastered several key writing skills – the ability to report on observation, analyze complex information, provide interpretation, use and document sources, and articulate and support a thesis-driven argument.  The exercise provides helpful feedback to students mid-way through their college careers. At another institution, the senior capstone experience challenges students to synthesize what they have learned and use higher-order skills in critical reasoning and writing to develop an original analysis based on extensive research.

 

Assessment can help institutions learn from one another.  Many colleges and universities participate in projects and consortia that collect survey data.  They do so to obtain general comparative data from similar schools because these data provide a context for their own results and allow them to assess their students’ responses over time.  Well-known examples of surveys include those conducted each year by UCLA’s Higher Education Research Institute, the Consortium on Financing Higher Education and the Higher Education Data Sharing consortium.  These data are shared across participating institutions in a way that facilitates analysis of longitudinal progress within a single institution as well as comparisons across peer institutions that can be used for bench-marking.

 

Assessment through standardized measures does not evaluate what is best and most valuable about American higher education. A liberal arts education requires faculty to take a holistic approach in judging whether students have developed both substantive expertise and the ability to give a clear exposition of original scholarship involving critical reasoning.  Based on our experience, we are skeptical about efforts to make this kind of assessment through standardized tests, including those that purport to measure critical reasoning.  While many of these tests incorporate exercises that aim to assess a broad range of student development, assessment experts are far from agreement about whether "value added" can be measured accurately across diverse institutions.  With students beginning their academic journeys at widely divergent starting points and pursuing a diverse array of curricula, we do not endorse any approach that depends solely on a single standardized measure or even a single set of standardized measures.

 

Assessment should be locally driven. We advocate locally-based, faculty-driven efforts to define and measure the skills and capacities that each institution emphasizes to meet its educational goals.  This approach will be more meaningful, and ultimately more effective, than any nationally standardized test. The examples illustrate some of the ways in which institutions are collecting data to help improve individual courses, set academic priorities, and evaluate whether specific innovations in the curriculum are having the intended result.  They are examples of how we believe assessment should be conducted – with local initiative, with the active involvement and support of faculty, and with an eye toward strengthening the mission of the particular institution.

 

Assessment is distinct from accountability; we want to emphasize this point made in the sidebar.  The principal purpose of accountability is to verify whether an institution has met the expectations of its various constituencies, and colleges and universities have an obligation to collect, analyze and provide data by which their constituencies may hold them accountable.  But the principal goal of assessment is to use various sources of evidence to make educational improvements.  Assessment may begin with the question of whether certain goals are being met, but it is ultimately about how to come closer to meeting those goals. Thus, assessment efforts should be squarely focused on improving the education of students and informing the institution’s efforts at self-improvement.

 

A Culture of Learning & Teaching

 

Many of our institutions have invested over a long period of time in complex, high-quality forms of assessment that include senior theses or other capstone projects, comprehensive oral or written exams in the major, writing portfolios, and significant investments in post-graduate outcomes research compared across institutions.  Our approaches to assessment encourage experimentation and exploration, and rely heavily on peer review.  Our peer review processes have created a system in which those best-suited to assess a given project do so, and our peer review systems have already incorporated new components of student learning research.

 

Our academic departments are made up of many master teachers with years of experience as well as creative junior faculty building their skill base.  Our systems of tenure and promotion encourage the communication of experience and knowledge across these networks through peer review of research and through course critique and mentoring within departments. We encourage institutions to enhance these systems to emphasize the best research on teaching and learning.  Our institutional leaders can strengthen these networks by providing information and resources through teaching and learning centers, faculty mentoring and enhanced access to support for innovative classroom experiments.

 

Importantly, the system of peer review should extend beyond our institutional gates into the accreditation system.  Just as our faculty members are most likely to use advice and criticism from experts in their own fields, our institutions must rely on the feedback of peer institutions that are similar in structure and mission in the accreditation process.  We believe the accreditation system works best when true peers review institutional progress, and we encourage accrediting agencies to match visiting teams thoughtfully and with the input of institutional representatives.  In order for this system to be successful, representatives from the full spectrum of institutions must make themselves available to serve on accrediting teams; we encourage our colleagues to recommit themselves and their institutions to such participation.

 

Conclusion

 

All of these efforts lead to an ultimate goal:  Advancement in our understanding of the work of educating students, and the continuous improvement that goes along with this understanding.  Assessment, if targeted and designed thoughtfully and responsibly in the context of our missions and goals, is an effective tool to encourage reflection and increased self-consciousness about what works best in educating the next generation of leaders. We reiterate our commitment to work for continuous improvement on behalf of our multiple constituencies, both within the United States and around the world.