Miller says he first developed the idea for Caesar in 2010, and that it was partially inspired by a crowd-sourcing system that CSAIL had recently developed called Soylent, which draws on crowds of people to help with editing tasks. Soylent and Caesar embody similar characteristics: Both take a large program, chop it into pieces, and distribute those pieces to a crowd for review and discussion.
“The on-campus problem we were trying to solve was slow feedback,” said Miller. “The students in our programming courses write a lot of code, and it takes a long time for a small staff of human graders to read and grade. The old way, it might take a few weeks to get feedback about what they’d written, and in that time they’ve written more programs—often repeating the same mistakes over and over.”
Because Caesar’s “reviewing interface” aspect is speedy, students are able to read their feedback prior to completing their next assignment—something that Miller believes will result in a better understanding of class material, and possibly higher grades. Also, by connecting students with multiple resources, students gain invaluable insight they might not have otherwise.
“Caesar substantially reduces the time it takes to get feedback,” said Miller. “Students typically hand in their programs on a Thursday, and by Monday they have comments about it from the crowd of reviewers.”
Miller believes Caesar can resolve similar problems in the larger context of online education programs, including massive open online courses (MOOCs).
“Online, the problem of scale is even greater—too many students, not enough staff,” he said. “The usual approaches to the scale problem in massive online education are purely automatic, which have a lot of limitations in the kinds of feedback they can give and the kinds of work they can evaluate. Caesar is [ultimately] trying to add a human dimension to the feedback.”
Miller has selected reputable sources that offer helpful feedback to students in his programming course.
“What we are trying to do is to learn how to use a crowd of people with mixed expertise in an intelligent way,” he said—“one that helps students and the ‘crowd’ expand their knowledge and improve on their expertise.”
Students enrolled in Miller’s online course, “Elements of Software Construction,” can benefit from the assortment of feedback that Caesar offers, because the nature of the course is so technical.
“Every time you interact with a new person, it’s an opportunity to learn, whether it’s a student demonstrating a new technique to a teaching assistant, or an alum providing a student with a valuable piece of industry advice,” he said.
Currently, Miller’s pool of resources is limited to members of the MIT community. He hopes to continue to expand this network in the future. He also has implemented tools for students and reviewers to interact further after feedback has been offered.
“Like Facebook and other social networks, Caesar provides opportunities for other kinds of interaction among its users,” the CSAIL release explains. “Reviewers can agree or disagree with fellow reviewers’ comments via and ‘upvote’ or ‘downvote,’ a process similar to the ‘like’ feature on Facebook, and can also leave comments for both students and other reviewers.”
(Next page: How Caesar is being expanded into other courses)
- Adaptive learning helps personalize instruction for students - March 29, 2013
- New software aims to improve course evaluation process - March 27, 2013
- Disability Resource Center empowers students to unlock their full potential - March 25, 2013