The MOOC metrics we should be tracking

Understanding student demographic factors might help us teach a new generation of learners how to succeed in MOOCs

An important technique for understanding what might really be happening in the world is to avoid being bowled over by stunning numbers.

For instance, when Massive Open Online Courses (or MOOCs – free classes offered by some of the world’s best-known colleges and universities) first made news, stories celebrating them as the solution to the planet’s educational ills focused on huge enrollments in the tens or even hundreds of thousands.

But soon critics of this new learning method started taking aim at a different remarkable figure: drop-out rates that typically topped 90 percent.

The problem with both huge enrollment and high drop-out figures is that they are based on the same inappropriate metaphor. For both assume that someone who hits the Enroll button on the Coursera or edX sign-up page is the equivalent of a student at a residential university committing to taking a course for the semester.

But MOOCs are offering a valuable product (a college course) for free. And, unlike a traditional college class, there is no consequence for signing up with no intention to complete (or even show up for) a class. This means a better metaphor for MOOC “enrollments” and “drop-outs” might be sign-ups and usage rates for any online service that attracts people willing to fill out a simple form to get something for nothing.

Which numbers are actually representative?

So if the most frequently touted figures related to MOOCs might not accurately reflect their success or failure, what numbers should we be using to evaluate their effectiveness?

Fortunately, the data generated by thousands of students taking hundreds of MOOCs over the last two years has given us a better understanding of what people do inside a massive online course, other than signing up and either finishing it (by earning a certificate of completion) or leaving before the end.

(Next page: Improving retention rates)