Study by Center Co-Directed by Professor Levin Finds MOOC Reality Not Yet Meeting High Expectations

The rollout of MOOCs, or massive open online courses, three years ago by some of the country’s leading universities triggered predictions that bricks-and-mortar campuses would soon be obsolete and that learning would be forever changed.

Well, maybe – but not any time soon, according to one of the first comprehensive studies of MOOCs from the perspective of institutions, just released by researchers at Teachers College. The study is based on 83 interviews with faculty members, administrators, researchers, and other actors in the MOOCspace from 62 institutions, mostly in the U.S. It includes 13 case studies to illustrate how MOOCs are successfully being used to address institutional goals.

The study finds that a primary goal for institutions offering MOOCs is to extend institutional reach and access to education. “MOOCs are providing educational opportunities to millions of individuals across the world,” write Fiona M. Hollands (Ph.D. ’03) and Devayani Tirthali (Ed.D. ’13, Ed.M. ’12), respectively of the College’s Center for Benefit-Cost Studies of Education and Institute for Learning Technologies, co-directed by Professor Henry Levin, in their report, “MOOCs: Expectations and Reality.”  However, to date, “most MOOC participants are already well-educated and employed.” Consequently, “the evidence suggests that MOOCs currently are falling far short of ‘democratizing’ education and may, for now, be doing more to increase gaps in access to education than to diminish them.”

Using MOOCs to “build and maintain brand” is another frequently mentioned institutional goal, but while MOOCs often generate media attention, “isolating and measuring impact of any new initiative on brand is a difficult exercise,” the report suggests. Indeed, increasing access to online offerings and enhancing brand may be contradictory goals, because the former can be seen as diminishing the selectiveness of the offering institution.

Hollands and Tirthali report that it is still too early to know whether MOOCs can live up to the hype of providing a cost-effective means for producing better educational outcomes on a mass scale. Cost analyses of MOOC production and delivery at four different institutions found that costs ranged from $39,000 to $325,000 per MOOC. “MOOCs have, so far, proved to be a significant drain on time and money for institutions,” Hollands and Tirthali write. That picture could change as institutions reuse MOOC materials, share them with each other, develop common courses, replace on-campus courses with MOOCs, and save on faculty teaching time and facilities costs. Revenue streams from MOOCs have been slow in materializing. Unless costs of MOOC production can be recovered through fees, Hollands and Tirthali speculate that “free, non-credit bearing MOOCs are likely to remain available only from the wealthiest institutions that can subsidize the costs from other sources of funds.”

As for improving learning outcomes, MOOCs, on the whole, cannot yet make that claim. “While interviewees provided many examples of how MOOCs have been used to change instruction, for the most part, actual impact on educational outcomes has not been documented in any rigorous fashion,” the report asserts. However, two cases highlighted in the report provide examples of positive effects on student performance as a result of adopting MOOC-inspired strategies such as frequent assessment and automatic feedback, or of integrating MOOCs into flipped on-campus courses. Hollands and Tirthali conclude that “while the potential for MOOCs to contribute significantly to the development of personalized and adaptive learning is high, the reality is far from being achieved.” To get there, “a great deal of coordination and collaboration among content experts, instructors, researchers, instructional designers, and programmers will be necessary.”

Hollands and Tirthali make several recommendations to institutions for increasing the value of MOOCs to improve access and educational outcomes, and to reduce the costs of higher education. These include:

 
• Identifying multiple channels of communication to advertise MOOCs to less connected audiences, and providing more instructional scaffolding to serve less educated participants
 
• Assessing the impact of MOOC pedagogy on educational outcomes by conducting pre- and post-assessment of participant skills and knowledge

• Developing metrics to assess gains in cognitive and non-cognitive skills that can be applied outside the MOOC environment
 
• Finding ways to confer economic value on MOOC completion, such as providing employer-recognized credentials
 
• Using MOOCs to substitute standardized courses across multiple campuses, or for continuing professional development and certification
 
• Working towards standardization of data formats across online learning platforms in order to facilitate research
 
• Establishing an accreditation system to evaluate MOOCs and other non-degree-based learning experiences to allow learners to accumulate a portfolio of credentials.

Hollands is the Associate Director at the Center for Benefit-Cost Studies of Education at Teachers College, Columbia University. Devayani Tirthali is a Research Associate at TC’s Institute for Learning Technologies.

The Center for Benefit-Cost Studies of Education (CBCSE) seeks to improve the efficiency with which public and private resources are employed in education. The center conducts research to determine the costs of educational programs as well as the economic value of program impacts in order to encourage educators, evaluators, and policymakers to consider these factors in conjunction with program effectiveness in addressing educational goals. CBCSE is co-directed by Henry M. Levin, TC’s William Heard Kilpatrick Professor of Economics and Education, and Clive Belfield, Associate Professor of Economics at Queens College, City University of New York.

Published Thursday, May. 15, 2014