Key points:
- Transformative technology can automate the creation of high-quality practice study questions
- Investing in learning technologies, not just teaching
- Rethinking education in a changing world: The power of online learning
- For more news on student learning, visit eCN’s Teaching & Learning hub
The Doer Effect is a long-standing principle that students who engage with more practice have higher learning gains than those who simply read text or watch a video. Originally published a decade ago, the Doer Effect has been a foundational principle in the learning sciences and the education industry; however, putting it into practice at scale has long been limited by the time and expertise required to author high-quality study questions. This study shows that AI-generated questions can help overcome that barrier, extending the benefits of the Doer Effect to more students.
During Fall 2022 and Spring 2023 semesters, researchers conducted a replication analysis using AI-generated study questions to help capture student performance data. The study was the first to replicate the Doer Effect using AI-generated questions, finding that students who answered study questions chapter-by-chapter increased exam scores by nearly 2 percent, which can be the difference between a pass or a fail for lower-performing students.
Engagement matters when it comes to learning
The research study included over 200 students in two course sections, with each asked to answer AI-generated questions embedded alongside textbook content in their digital learning platform. Students in the first group had until the end of the term to complete the required questions, while the second group had to complete the required questions chapter-by-chapter with multiple due dates before the final exam.
The students who were required to complete formative practice chapter-by-chapter had only 6.8 percent of their attempts on material not relevant to the upcoming exam, but those students who were only required to complete practice questions before the end of term had a staggering 40.3 percent of irrelevant attempts. This dramatic difference validated the understanding that students who practice as they go concentrate better on material directly aligned with their exams.
Exams scores improve with the Doer Effect
By actively engaging with study questions throughout the term, students achieved improvements in learning outcomes. The students who practiced chapter-by-chapter saw average scores increase by nearly two percentage points, with significant gains observed among lower-performing students. While the mean scores increased by two percent, increases for students with grades on the cusp are the most notable. At the 25th percentile, exams moved from 71 percent and 72 percent to 74 percent and 76 percent respectively, meaning a grade shift from C- to C. Similarly, the 50th percentile on those same exams saw a shift from 78 percent to 80 percent for each group, meaning a C+ to B-. For students struggling with their GPA, utilizing practice questions throughout their course work can potentially push them up a letter grade.
Benefits beyond student outcomes
For complex courses, educators often prepare study question guides for every chapter in the required textbook to help students learn the material. This practice is beneficial for students, but time consuming. One way to ease this burden on professors is to embrace the shift to AI and provide guidelines that have the student’s best learning interest in mind. AI-generated study questions like those used in the study are integrated into the textbook, saving educators time, increasing engagement and driving positive results.
Simple changes in how learning is implemented make a big difference in how students engage and benefit from their courses. During a time when the value of higher education is being questioned, institutions must learn to strike a delicate balance between course policy, engagement, outcomes, and learning theory to ensure they’re not only providing a great education, but the skills and experiences needed for learners to thrive post-graduation. By leaning into today’s transformative technology, faculty and administrators can automate the creation of high-quality practice study questions, giving students the tools they need to perform better in class while easing the time and preparation needed to get them there.
Rachel Van Campenhout, Ed.D, is a Learning Scientist at VitalSource.