The traditional educational model tends to be “learn, then test.” Tests are perceived as unpleasant, often high-stakes experiences that are separated from learning. What if we used the principles of digital transformation to rethink the next generation digital learning experience? If assessment was continuous and embedded in our learning, would learning be more engaging, personalized, and effective?
We believe assessment and diagnostics should be part of higher education’s digital transformation. We can use technology to obtain more holistic information about a learner by creating learning and testing opportunities through interactions in games, simulations or virtual reality. A learner can fly the plane, operate on the patient, drive the car, learn and be tested all in one. This type of continuous assessment is enabled by the collection of rich data—one of the key elements of digital transformation. Computational models are fit to that data in order to provide feedback to the learner and to feed the learner forward to the next learning opportunity.
Learning in an active, immersive environment guided by continuous feedback can lead to a deeper understanding of the material, improved ability to apply the information, and confidence in its use. These environments are being designed to provide ample opportunities for learners to prove their mastery of the domain. Assessments for skills such as collaborative problem solving can be presented as challenges or quests with the team’s and individual learners’ performance scored on the evidence provided. A multi-sensory experience, such as for medical practice or for piloting a jet, can generate multimodal data which may include performance tasks that have been recorded as video and audio, or eye-tracking data that are then analyzed using machine learning techniques. See, for example, the CPS X Collaborative Game from ACTNext.
Simulations can collect hundreds of thousands of data points, such as the order of steps taken in solving a problem or the time spent making a decision. These data can be used by algorithms that predict the learner’s aptitude to learn, critical thinking skills, and other attributes. A blend of psychometric theory, artificial intelligence algorithms and machine learning can measure latent student abilities; we call this computational psychometrics. In this scenario, assessment is embedded in the learning experience—it is not separate from it.
Even difficult to measure skills, such as creative thinking, can be assessed with the help of technology. For example, PISA (Programme for International Student Assessment) 2021 will assess creative thinking using simulations that elicit the rich evidence that is needed to support the claims of whether students show creative thinking skills. Creative thinking is a necessary competence as society is increasingly dependent on innovation. The impact of assessing creative thinking may go beyond assigning a score—it may catalyze better instruction. As routine human tasks are automated by AI or robots, creativity becomes even more important. For educators, digital transformation should encourage us to focus on developing uniquely human skills.
We typically think of learning and testing taking place within the context of a course. However, the Internet has accelerated our ability to learn at any time and from multiple sources—what is called self-directed learning. A personalized learning pathway might guide learners independent of a course, school or curriculum. An example is ACTNext’s Educational Companion App. Users access the app on a phone. The structure is based on ACT’s Holistic Framework that includes core academic skills (e.g., mathematics), cross-cutting capabilities (e.g., problem solving), behavioral skills (e.g., open mindedness) as well as education and career navigation skills. The app is powered by a Recommender and Diagnostic (RAD) engine that is delivered via an API. It identifies the strengths and weaknesses of each student, provides feedback on current capabilities and guides the user to open educational resources that can augment their development. The application integrates a number of factors that determine success, both academic and social-emotional ones, enabling users to manage their development from school, to college, to career.
What role do assessments play in the higher-ed digital transformation?
These unique pathways to success may become even more important as individuals progress through their careers. Our professions are constantly being reshaped by new technologies and emerging processes, so we must continually learn new skills or upgrade existing ones. The personalized feedback provided by the Educational Companion App is built on an analysis of an individual’s skills, behavior, and means of knowledge acquisition. It can provide a guide for the long-term, helping us adapt to new roles or transitioning from one career path to another.
As mentioned above, data plays a crucial role in any digital transformation, including that of assessment. In the era of big data, the expectation is not only that we have access to large volumes of data, but also that the data are matched and can be aligned and analyzed on different dimensions in real-time. Unfortunately, that expectation does not align with today’s reality. To realize the benefits of digital transformation, educators will need to agree on data standards and adapt today’s systems to data cubes and data lakes. The work will not fall to data scientists, alone. In a world of big-data and algorithms, educators too will need to become data-literate or data savvy, knowing how to effectively use and critically evaluate data and its sources.
Assessment can and should be part of every higher-education institution’s digital transformation.
New tools and techniques are being developed that can truly transform teaching and learning, thereby having a significant impact on individuals and society. Our belief is that the process of testing and assessment can be positive experiences that support and guide learners throughout life.