In his essay, the President of the Council for Aid to Education (CAE), Roger Benjamin, discusses how a standardized assessments initiative could provide a great tool for decision-makers, as long as:
3. Validity and Reliability are Ensured.
“Measurement scientists have developed criteria to evaluate the validity and reliability of assessment protocols,” explains Benjamin in his essay. “Validity refers to the extent to which the test measures the knowledge, skills, and abilities it was designed to measure. Reliability is the degree of consistency of student’s (or institution’s) scores across a test’s questions, the consistency of scores across different assessors, and whether the tests are given to students under the same conditions and over the same time period. Standardized assessments that provide statistical evidence of reliability on these criteria are preferred. Moreover, measurement scientists are insistent about recognizing this point for any tests that have stakes attached to them.
Boards of trustees and administrators want to know how well their institution is doing compared to institutions that are similar in student characteristics, financial support, size and other characteristics. Therefore, it is essential that test data for these purposes be based on the transparent criteria measurement scientists have developed for standardized tests.”
4. Researchers and Practitioners Work Together.
“…faculty must be partners with their measurement scientist colleagues in providing content for the design of standardized test items, evaluation of the standardized test results, and the development of formative test items that are aligned with the standardized tests,” writes Benjamin.
“Measurement scientists, the statistical-based tools they use, and the test analyses they produce are often challenged by faculty. Why? Faculty are housed within departments that are granted relative autonomy by the university to recommend what to teach, who to teach it, and how students should be assessed. Education assessment test results and analyses are typically isolated, one-off research activities that are not related to either faculty engaged in teaching or to researchers in other fields relevant to improving student learning. Independent experts, no matter how talented, are not considered to have the standing necessary to contribute to department affairs. However, measurement science, including its education assessment sub-groups, is a branch of statistics that has been in good standing in the Academy for hundreds of years.”
5. Everyone Works Toward an Integrated, Interdisciplinary Approach.
“The premises of the value system of science, peer review, transparency, and the ability to replicate results are familiar to faculty and administrators,” Benjamin notes. “Most faculty should and will accept assessment-related work based on these core principles. When paired with a coherent and compelling use-inspired basic research strategy, it is possible to imagine a more integrated, interdisciplinary approach to the challenges that higher education faces.
Already, an initial effort has been launched to include all subjects within standardized assessments. The Gates Foundation’s Measuring College Learning (MCL) project is a collaboration of six national disciplinary associations to define the core learning outcomes of their fields. Prospects for success of this endeavor are good. If this group of six associations succeeds in creating attractive, reliable and valid tests, other disciplines will follow. It will be important to develop standardized tests for the arts and sciences that form the basis for general education curriculum; other professional schools and applied subjects should and will follow.”
[Read Benjamin’s full essay with more thoughts on this subject here]
- 25 education trends for 2018 - January 1, 2018
- IT #1: 6 essential technologies on the higher ed horizon - December 27, 2017
- #3: 3 big ways today’s college students are different from just a decade ago - December 27, 2017