The pedagogy behind skills-based assessment

Key points:

• Without meaningful, verifiable learning taking place, microcredentials hold no value or credibility
• Effective skills-based assessment, or simulation-based assessment, is based on a strong competency framework
• See related article: Employer demand for microcredentials is on the rise

Increasingly, colleges and employers are looking for strategies that will allow them to adopt skills-based learning and assessment, either as part of microcredentialing or a corporate training and development program.

Online learning platforms such as Udemy, Udacity, and Coursera have expanded rapidly by making micro-learning and training more accessible and affordable for a wide population. However, as online learning has become more popular, some providers simply convert print training materials directly to digital and try to use the same strategies that are used in a traditional classroom within the digital environment. 

A more effective approach is to ensure that those courses and assessments are built on sound pedagogical practices that actively help learners actually learn the content they are supposed to be mastering. Without meaningful, verifiable learning taking place, those microcredentials hold no value or credibility.

From learning goals to interactive simulations

For skills-based learning and assessment to be effective, they need to work backwards from the learning goals to identify and map out an authentic experience in a digital environment.

Tara Laughlin, senior education designer with Education Design Lab, and the lead on the XCredit Project, whose mission is to signal to employers the skills that job seekers have attained informally on the job or in life, said that skills-based assessment is a new frontier: “It’s an interesting combination of things that there isn’t a precedent for. The Lab looked at simulation-based, digital assessment, and competency-based assessment, and synthesized together learning from across all those fields and approaches to inform the approach we’ve taken with these assessments in the XCredit project.” 

Shaping a real-world model for digital learning starts by tapping into the students’ existing mental model of the topic, including their understanding of any interrelated functions. The process begins with assessing students’ ability to apply that knowledge. Interactive simulations provide an authentic environment that allows students to apply the knowledge and course-correct immediately within the simulation, providing closer mapping between the practice and the real world. As a result, there is increased transference of those skills when students actually encounter the situation in the real world. Interactive simulations must be developed by first declaring the desired student outcomes that the educator is targeting and keeping those outcomes in focus throughout the development and assessment process.

Take, for example, the Ps of marketing (Price, Product, Place, and Promotion). In order to learn those skills, students might run a virtual food truck. They decide where to place the food truck. The demographics of that area may then affect other factors, including:

• Price (what people are willing to spend within that area);
• Menu/Product (what items people in that area are most likely to buy); and
• Promotion (how to interest people in the food truck and what it has to offer).

If they complete those tasks properly, they are successful. If not, they are coached along the way on how they can perform those tasks better next time, which can help set them up for future success. The story-based experiential learning style helps students remember that content longer and makes them feel more prepared to utilize those skills in the real world, which is deeply empowering. 

Educators can also build on those models to test other skills. For example, an angry customer could come to the food truck counter, which gives the learner the opportunity to practice other valuable soft skills.

Laughlin pointed out that the process of taking a simulation-based assessment has a parallel to getting your driver’s license. “You do the written test, and you might score 100 percent on that. And it’s largely recall, knowledge, base things that are important to know, right? What do you do when you come to a red light? You should stop. But that still doesn’t mean that when you get behind the wheel of the car, you can actually pull that knowledge together in such a way that you can act on it.” A simulation-based approach, on the other hand, gives drivers the chance to test their skills more thoroughly in a safe environment, which puts them in a better position to put those skills into practice. 

Skills-based learning and assessment and simulations offer a number of other key benefits, including: 

• Students can’t game the system as easily with skills-based assessments and simulations as they can with typical multiple choice-based assessments. Learned test-taking strategies don’t work within simulations, which means students must actually learn the material.
• Unlike tests where students know there is a right answer, simulations provide an opportunity to react as they would in a real-world scenario where choices aren’t black-and-white, which improves overall learning and cross-domain transference.
• Automated assessment of simulations saves educators time that they would otherwise spend grading exams or essays.

Building simulations

The bedrock of effective simulation-based assessment is a strong competency framework. First, content creators look at the high-level competencies learners need to accomplish a set task. Then, they break each one down into its component parts. The goal is to break each competency down into observable, demonstrable behaviors, then create a setting in which students have the opportunity to express their competency and learning in those areas. Content creators must make sure that the framework is aligned with research in that subject matter and validated by different stakeholders, including subject matter experts and a team focused on equity.

There is a storytelling aspect to the work as well, as Laughlin noted. “We tend to break the assessments up mentally into acts or chapters in our heads” so that simulations, like screenplays, often have “a three-act structure.” that learners are either consciously or subconsciously familiar with. 

In the end, said Laughlin, simulation-based assessments are the closest thing students can get to real-world experience. “The research shows there is a gradation, even within performance-based assessment, of how you can increase the authenticity and transferability of the finding. Simulations fall in one of those highest tiers in terms of real-world application, but also the likelihood of the skill that’s being demonstrated transferring to another context.”

Related: Biggest barrier to skill development is cost, survey finds