Review of Bill & Melinda Gates Foundation Digital Courseware Investments offers findings and lessons learned
[1]A new report assesses five years of technology investments in digital courseware by the Bill & Melinda Gates Foundation Postsecondary Success initiative [2] in order to understand what is required for digital courseware to produce positive student learning outcomes at scale.
The report comes from SRI Education, a division of the nonprofit research and innovatgion center SRI International.
At the request of the Gates Foundation, SRI Education reviewed 137 postsecondary online and hybrid courses and provided a synthesis of the findings, along with implications and recommendations for future investments in learning software for colleges and universities.
“Digital courseware has the potential to improve student learning outcomes and catalyze changes in education practice. These two affordances align well with the Gates Foundation’s emphasis on improving college success for underrepresented minorities, low-income students, and first-generation college attendees,” said Barbara Means, Ph.D., director, Center for Technology in Learning, SRI Education.
(Next page: Findings from the report)
“With this in mind, we looked at online and blended learning courseware in three major areas–context of use, instructional and technology design, and implementation practices. Our goals were to determine courseware features associated with greater learning effectiveness and to provide guidance for funders as they look to make learning technology investments in the future.”
After reviewing the major Postsecondary Success projects, several features were identified as having positive effects on student learning:
- Instructional designs or redesigns for entire courses produced significantly greater learning effects compared to less intensive approaches such as designs for supplemental resources.
- Learning effects from digital courseware implementation were greater in community colleges than in four-year colleges.
- Courses in mathematics produced more positive learning effects than courses in other subject areas such as science or the humanities.
- Course implementations using individualized pacing had more positive impacts than those with class-based or a mixed form of pacing.
- Online courses in which students’ dominant role was solving problems or answering questions had more positive effects than those where most of the students’ time was spent reading text or listening to lecture videos.
- Adaptive learning technologies demonstrated larger learning effects than non-adaptive technologies.
Along with these key findings, SRI researchers describe lessons learned from their analysis of the Postsecondary Success portfolio. For example, the needs of local students and the institutional context are critical factors in the successful adoption of existing digital courseware into higher education classes. Courseware innovations that have been successful in one campus often fail to achieve similar results when implemented in new settings. Moreover, while faculty are often willing to try courseware developed at another institution, they want to be able to modify it for their own circumstances, and often the courseware that interests them is not easily modifiable.
Additionally, courseware innovations that improve student learning and course completion rates are developed through multiple cycles of design, testing and revision. Data that link student outcomes to specific design features and implementation practices need to be systematically collected and analyzed to gain insights on how to improve software design and implementation practices.
The report includes several recommendations for organizations looking to make future investments in digital courseware to enhance student outcomes.
- Funders should consider investing in high-quality courseware for lower-division courses that is designed for reuse, as well as in additional research on effective strategies for implementing the most effective courses at scale.
- Courseware funders should consider enlisting the help of third-party evaluators who use consistent measures and methods to collect and analyze data that can provide objective evidence of effectiveness.
- Funders should take a phased approach to supporting courseware innovations, with later stages of funding dependent on demonstrated capacity to collect data that can inform improvement.
- Part of the funders’ investment in evaluating courseware and technology tools should be devoted to examining longer-term impacts and implications for degree completion.
Material from a press release was used in this report.