Penn State uses hard data to see if faculty professional development in student engagement actually makes a difference.

faculty-student-engagementHow can institutions know whether or not professional development (PD) programs for faculty are effective? And does student engagement increase when faculty complete PD courses aimed at promoting student engagement in the online learning environment? According to one large university, PD in student engagement makes a significant difference…when applied in practice.

Recently, Penn State’s Online Campus set out to help answer the questions above as part of a pilot study within a large online Bachelor of Science in Business program in the spring and summer semesters of 2014 that included 2,296 students.

The pilot study aimed to measure, through student responses to qualitative, open-ended questions, whether or not faculty who had PD on student engagement made a difference in the level and quantity of student engagement post-PD.

The study addresses these questions in an “effort to think about ways to measure whether and to what extent faculty are applying the material taught in two [PD] courses offered by the Faculty Development Unit at the Penn State World Campus,” notes the report. “The mission of this unit is to support faculty in best teaching practices in order to positively impact student success.”

Though the report’s authors emphasize that the survey response was low (only 159 student scores could be used to answer the research questions)—and hope to increase the response rate in the future—the survey still revealed that 75 percent of respondents who reported that their courses had engaged them were taking courses taught by instructors who had taken one or more of the PD courses offered by the University.

However, 25 percent of the students who had instructors with PD were not engaged in their courses. The majority of student respondents who felt this way cited lack of engagement related to lack of instructor participation and feedback on assignments and/or discussion posts.

Yet, the students with instructors who had not taken either of the PD courses and who reported being disengaged in their courses attributed their lack of engagement to reasons similar to those given by students who were disengaged in courses where the instructor had taken one or more of the PD courses.

Aligning PD to students’ feelings of engagement

Beyond simply asking students whether or not they felt engaged, Penn State Online also asked students to describe the specific actions or course designs used by their instructors that made them feel more engaged with the course.

(Next page: Student engagement broken down)

According to the survey respondents:

The majority of students felt that engagement meant interacting both with their peers and with the instructor, participation in online discussions, and involvement with the subject matter. Students often mentioned that timely, meaningful instructor feedback was a part of those interactions, says the report.

“Engaged means to me that I am learning from the instructor, the material, and most importantly, from fellow students,” said one respondent. “The exchange of ideas and experience is most helpful.

Students part of the survey also indicated how important relevance and real-world application was in their course.

“This course has all the right stuff, just lacks us thinking more about current events,” said one student. “I really think this course could embody engagement if we didn’t spend some weeks thinking about the basic concepts by themselves and instead thought about them as applicable to the real world and what is going on around us.”

However, students also noted that it’s not just the course content that keeps them motivated to learn, as instructor enthusiasm was heavily emphasized.

“To be ‘engaged’ in a course is to have a desire both to actively participate in the course work and desire to extract more than is offered through the standard assigned course work,” said one students. “Please note, the word ‘DESIRE.’ The instructor can easily encourage students to extract more information on a small amount of enthusiasm and encouragement. If the instructor has a cold feel when interacting with students this often closes the door for ‘the desire to learn more.”

Students were also asked whether the course they had taken met their definition of engaged learning, not just the University’s. Over 75 percent of students surveyed felt that the courses met their definition of engaged learning, which included an emphasis on personalized feedback.

“The instructor provides meaningful and personalized feedback to my assignments which assist me in determining whether or not I’m on the right track,” noted one student. “It’s not about just getting a good grade and degree for me I genuinely want to learn and have the experience I would have if I were attending class on campus.”

However, some students said they felt engaged not because the instructor had participated in the course, but, again, because of the applicability of the course materials to real-world experiences.

“I said yes because the three projects/assignments/labs which required students to get out there, were really good (if you challenged yourself),” said one student. “The rest of the work could have used more effort from the professor. What is the point in having discussion boards if we do not discuss?”

According to the report, for students who said that their courses did not meet the definition of engaged learning, “the overwhelming consensus was that their instructors were not present in the courses and were late returning graded assignments.”

“We did have discussion posts but nothing that pertained to the class. We had a single group project. For comprehensive problem five, both the teacher and tutor had told the class ahead of time that this was an assignment that most students have difficulty with. If that is the case and instructional staff knows that, I would think that they would have had a blackboard elluminate session or at least more interactive session than what we got, which was ‘Read the text very carefully,’” lamented one student. “In online courses so much depends on the student’s reading comprehension and we accept that going in but we could use some teacher time…out here, we often feel alienated and as though we are left to our own devices.”

“According to the accounts of the students surveyed in the online business program, it was evident that most instructors with relevant professional development applied strategies to engage the students,” concluded the report. “Both quantitative and qualitative data supported the fact that many trained instructors had a positive impact on student engagement…The qualitative data also provided evidence that trained instructors were more actively involved in the courses and provided more timely and meaningful feedback than did untrained instructors.”

Moving forward, Penn State Online’s research team plans to target additional programs in different disciplines as well as programs at the graduate level.

“Although some instructors who had taken either of the professional development courses did successfully engage students, it would be interesting to determine the extent to which this occurred using a larger sample,” noted the authors. “On this point, we do not know whether the non-trained instructors were able to engage the students because these instructors were experienced in the face-to-face and/or online context, because they were trained in pedagogy elsewhere, or for other reasons. There is also the possibility that some instructors are natural-born teachers.”

For more in-depth information on the survey, its results and methodology, read the full report “Measuring Student Engagement in an Online Program.”