What should MOOC quality standards look like?

Research delves into issues when trying to gauge instructional and design quality in MOOCs for credit.

[Editor’s note: We originally reported that QM was under iNACOL, though it is a separate set of standards.]

As more institutions consider offering MOOCs for credit, often the MOOCs provided by third-party platforms, researchers say it’s imperative to gauge instructional and design effectiveness…but how, and with what quality standards?

These are the main questions posited by Patrick Lowenthal, assistant professor at the Educational Technology College of Education at Boise State University; and Dr. Charles Hodges, associate professor  of Leadership, Technology & Human Development at Georgia Southern University, in their research study on trying to measure the quality of MOOCs.

The researchers explain that as institutions look to third-party (Coursera, edX, Udacity, etc.) provided MOOCs for credit, it’s important to investigate whether these MOOCs meet certain standards of quality. The problem is, very little research has been done to investigate the instructional quality of MOOCs.

“Due to this problem, we decided to investigate the design of MOOCs as determined by certain, accepted online course quality frameworks,” write the authors in their study. “We began this study with an assumption that MOOCs, just like formal online courses, are not inherently good or bad. Further, there are likely some things that members of the academy can learn (both good and bad) from analyzing MOOCs.”

Where to Start?

Though there are many quality assurance programs for online courses, such as California State University Chico’s, or the Online Learning Consortium’s (OLC) Five Pillars, Lowenthal and Hodges decided to use Quality Matters (QM), since it is “one of the most popular and widely used quality assurance frameworks in the U.S…there are currently more than 800 QM subscribers.”

The study notes that QM is a peer review and faculty development process that is centered on eight general standards (each of the standards has a number of related and more specific sub-standards):

• Course overview and introduction
• Learning objectives
• Assessments and measurement
• Instructional materials
• Learner interaction and engagement
• Course technology
• Learner support
• Accessibility

In order to pass QM’s standards, a course must receive an 85 percent and meet all of the essential standards. Yet, none of the six MOOCs chosen (2 from Coursera, 2 from edX and 2 from Udacity) by the researchers passed the initial review. However, one Coursera MOOC and one edX MOOC scored very well overall in terms of total points. The Udacity courses (both self-paced and offered year-round) performed the worst.

“The fact that all six MOOCs did not pass a QM review does not suggest that all of these MOOCs were poorly designed,” emphasize the researchers. “In fact, all six MOOCs met…specific review standards.”

For example, Lowenthal and Hodges found that all six MOOCs studied introduced students to the purpose and structure of the course; had the instructor provide an appropriate and available self-introduction; provided assessments that measure the stated learning objectives and are consistent with course activities and resources; provide students with multiple opportunities to measure their own learning progress; employed accessible technologies and provided guidance on how to obtain accommodation; provided course tools and media to support student engagement and guide the student to become an active learner; etc. (For a complete list of all the standards and sub-categories met, read the full study).

(Next page: MOOC quality standards failed; challenges to MOOC quality standards)