Technology key to analyzing assessment data

Through the careful application of technology in classroom assessments, schools and teachers can improve instruction for students, and states can develop comprehensive longitudinal data systems to better analyze student performance, according to data in a new report from the State Educational Technology Directors Association (SETDA).

"Technology-Based Assessments Improve Teaching and Learning," part of SETDA’s Class of 2020: Action Plan for Education series, highlights innovative approaches in assessing student progress, and suggests that using technology to assess students in a less formalized manner can be extremely beneficial, not only for students, but for teachers.

"Our educational system can learn from business and realize the incredible return on investment inherent in building smart database and assessment systems using technology to analyze each student’s progress in a timely, personal and relevant manner," the report says.  "Additionally, if done with interoperability and alignment in mind, student data can be generalized and privatized to be used in the state systems to inform policy and school reform efforts."

Of course, technology alone is not the solution.  Adequate teacher training, IT support, and strong leadership are essential in ensuring that data analysis drives instruction at the school, district, and state levels.  More personalized testing, without increasing the number or frequency of tests, can have a positive impact on assessing standardized testing requirements.

SETDA recommends that school and district leaders incorporate "innovative, consistent, and timely assessments into daily instruction;" as well as making sure that sufficient technology infrastructure and support is in place.

Barriers to conducting formative assessments vary among schools.  Many districts lack sufficient technology, including broadband internet access, for broad-based formative assessment implementation, according to the report.

Inadequate teacher training leaves many teachers feeling unable to analyze the data produced in their assessments.  In addition, many teachers feel pressured to cover state- or district-required material in light of the federal No Child Left Behind Act (NCLB) requirements, leaving them little or no time for formative assessments or to re-teach elements of the curriculum that might require remediation.

Training teachers in the proper uses of data will go a long way in ensuring that each child’s potential is reached, and using technology and formative assessment to strengthen the home-school connection by communicating with parents on student progress is strongly recommended in the report.

Schools should make sure the data flowing into classrooms are user-friendly, timely, and accurate.  Students and teachers will benefit most from technology when computers and other technologies are used continuously and seamlessly in instruction and assessment.

States are required to administer at least one end-of-year test in math and reading in elementary, middle, and high school, and NCLB mandates that states set individual measures of progress, Adequate Yearly Progress (AYP), with a goal of making all students and schools proficient in all testing areas by 2014. 

Despite those testing requirements, few states have aggregated testing data into a comprehensive statewide longitudinal system.

A 2007 survey by the Data Quality Campaign (DQC) found that only 16 states have at least eight of 10 essential elements, as identified by DQC, needed to develop a comprehensive longitudinal data system.  DQC advocates the implementation of state longitudinal data systems to improve student achievement and is managed by the National Center for Educational Achievement and supported by the Bill and Melinda Gates Foundation.

Four states– Arkansas, Delaware, Florida, and Utah–reported having a state data system that includes all 10 essential elements of a longitudinal data system.  Three states–Idaho, Maine, and Maryland–have just three or fewer essential elements.  Thirty-six states have built or are planning to build data warehouses using web-based data analysis and reporting tools that make these data accessible and user friendly.

The report recommends that each state redefine its role of "Data Compliance Officer" to "Data Leader," using data to improve education at all levels.  States also must begin helping schools and districts address how relevant formative assessment and demographic data can "flow-up" to the state to inform systemic changes in policies regarding school reform and student achievement.

The report touches on both summative and formative assessment, but focuses on formative assessment’s importance. 

Timely information about individual students is a critical component for improving the way that teachers teach and students learn.  Students should continually learn and progress throughout the school year–within a lesson as well as between lessons.  When teachers have access to data to regularly assess student progress, they are able to individualize instruction for each student. 

Because formative assessment, ideally, occurs throughout the learning process and gives teachers and students ongoing, timely feedback, the resulting individualized instruction lets students with different learning styles succeed in the same classroom. 

"Administrators cannot wait for the annual results from state standardized tests to see if students are mastering the required concepts and whether they are achieving their school improvement goals," the report says, adding that formative assessments are not about high-stakes accountability testing, but instead are about individualizing instruction so that all students achieve their highest potential.

Current and emerging assessment trends identified in the report use different state initiatives as examples of how different technology-based assessments are helping to make states more efficient.

Schools and school systems are using online assessments through secure network connections to assess student understanding of content regardless of the delivery methodology.  Online assessment is being used for a variety of purposes, including low-stakes testing that provides feedback to the student or teacher, in summative context for student grade promotion, and, in some cases, to facilitate state standardized achievement tests.

Virginia’s web-based Standards of Learning (SOL) Technology Initiative began with the goal of having schools use web-based systems to improve the instructional, remedial, and testing capabilities of the state’s SOL achievement tests.  Virginia hopes to administer all SOL tests online by 2009, and ihas dentified benchmarks schools must meet in order to administer an online testing program. 

Each school must give students access to computers at a ratio of one computer for every five students, create an internet-ready local area network, and ensure high-speed capabilities.  By delivering SOL tests online, the state’s administrators will be able to take advantage of web-based result reporting, leading to reduced turnaround time when waiting for student results, leading to potential increases in instructional time.

Many states, districts, and schools are using project-based assessments either on their own or embedded in core subject-area assessments to test competency in certain skills identified by the Partnership for 21st Century Skills (P21) as essential to a 21st-century education, such as learning and thinking skills, global awareness, and information and technology literacy.

The National Assessment of Education Progress (NAEP) developed a pilot test, Problem-Solving in a Technology-Rich Environment, to test students’ mastery of 21st-century skills within a physical science assessment.

During the assessment, students were given two extended scenarios that would measure their ability to solve problems using technology. The assessment required students to search the internet, using a simulated web environment, and locate and synthesize information about scientific helium balloons.  The "Simulation" scenario required students to conduct experiments of increasing complexity about relationships among buoyancy, mass, and volume.

These scenarios were delivered via school computers or on laptop computers taken into the schools. The purpose was to use technology to measure skills that cannot be easily measured by conventional paper-and pencil means. The assessment produced a total score and separate scores for scientific inquiry, computer skills, scientific exploration, scientific synthesis, and computer skills.

"Given the way that technology can now alter the speed and location of assessment, many options now exist to embed ‘on the fly’ assessment into curriculum content and lessons.  The days of hand-written records and paper copies of classroom assessments are quickly fading into [memories of] the 20th century," the report says.

"With the increased curriculum content to cover, teachers do not have the time to use paper methods for formative assessment when there are a wide variety of technology tools available," such as handheld devices, personal electronic response systems, and software.  

Blogs, chats, and Wikis used in the classroom help teachers gain an understanding of what students know or don’t know. Many districts and states are using technology-based programs and systems that give teachers formal and informal assessments to track student progress weekly, even daily. These types of formative assessments help keep students on-track with achievement, while also providing opportunities for students to participate in engaging activities based on abilities and needs. 


Technology-Based Assessments Improve Teaching and Learning

2007 NCEA Survey of State P-12 Data Collection Issues Related to Longitudinal Analysis