What one university learned after 15 years moving evaluations online
Student evaluations of courses and the faculty that teach them are an important aspect of quality control at colleges and universities.
How a course is taught, whether it will we be taught again, and even if the instructor teaching it will have a job next year can rely on what students say on these short surveys — questionnaires that are often hurriedly filled out at the end of a class, usually with finals week or a school break looming.
Not surprisingly, many faculty and administrators aren’t huge fans of the results. But one university has spent the last 15 years trying to improve the system, and in the process has taken its course evaluations online.
(Next page: What took so long?)
“There was some dissatisfaction among the faculty with the way evaluations were being conducted,” recalled Phil McCartney, one of the minds behind the revamp. “We had a feeling it could be done better.”
McCartney is a member of the Teaching Effectiveness and Enhancement Committee at Northern Kentucky University. In 1999, he was a member of a small group that tried to overhaul course evaluations, starting off by putting together a report of faculty suggestions.
One of those ideas was putting the evaluations online. This could allow students to take the evaluations at a different pace, and allow for questions to be more customized. Changing up the questions was an important part of what McCartney and his colleagues hoped to accomplish, he said.
“We didn’t just want to get answers to the same question,” McCartney said. “We wanted different, better questions, and to get more thoughtful questions, and not have a system that would go unchanged for years and years.”
Given the feeling toward the old-fashioned surveys, one might assume the project would be fast-tracked after the report. Instead, finding the right methods and then convincing faculty of those changes took more than a decade.
“There was this general paranoia at the time about the technology,” McCartney said.
Kenneth Rhee, another member of the TEEC said the paranoia might have went even deeper than that. Many faculty did not like the old evaluations, but they had grown accustomed to them after more than 30 years of use.
“For some of those who were opposed, yes, it was about the technological change, but a lot of the obstacles came down to figuring out how we could most effectively carry out an organizational change,” Rhee said. “We were asking people to do things in a really different way.”
(Next page: Keys to successful launch)
By keeping the faculty involved in every step of the long process, the team was slowly able to change minds and get the majority of them on board. The new online, customized evaluations were finally launched in 2011.
That culture of involvement, Rhee said, has also been extended to the students who fill out the new evaluations. Faculty and administrators play up the importance of evaluations from the first day of class; the surveys aren’t relegated to a footnote in the last week.
“We ask faculty to put a statement in their syllabus saying how important evaluations are,” Rhee said. “They explain the difference between asking for feedback and valuing feedback.”
And that, combined with the ease of going online, has seemed to work. They surveys now have a response rate of 70 percent.
“We have a student code of conduct, and as a part of the discussions, we talked about the meaning of what it is to be a citizen of the university,” McCartney said. “We ask them to think about the long term-relationship with the university and what it means to be engaged here.”
But the TEEC’s work with evaluations isn’t done yet, Rhee said. Getting the surveys online and students and faculty to engage with them was just a 15-year first step.
“You cant just say we have evaluations online, the technical problems are solved, and we can wash our hands of it move on,” Rhee said. “We have to figure out who all should have access to this data. How is it going to be utilized? You have to monitor the nitty gritty. Should we do all of this in-house? Go to an outside company? Are their security issues there? To be effective, you need people minding the store.”
Follow Jake New on Twitter at @eCN_Jake.