Voice from the Field

What Is the Right Test?
Toward Individualized Instruction: The Evolving Nature of Assessment

A BLEgroup Blog — Friday, December 09, 2011

By:
Eliot Levinson, Ph.D., CEO, BLEgroup
Don Hall, Director of Product Evaluation, BLEgroup
Kenneth Eastwood, Ph.D., Superintendent, Middletown City Public Schools, New York
Donna Wright, Ph.D., Deputy Superintendent, Knox County Schools, Tennessee

Analysis of Evolving Assessments

More than 50% of students entering high school are two or more years behind in at least one subject on meeting the academic grade-level standards, less than 40% are proficient, and only 5% are advanced. What do we do about the more than 50% of students who are missing the necessary prerequisite skills to master their current standards?

Good instructional leaders have a vision of individualizing instruction to meet the needs of individual students. Unfortunately, they lack the assessment and diagnostic tools to make this vision a reality. Assessments are the tools used to address student improvement. They range from “thermometers” that tell us what standards a student has mastered or not mastered to diagnostic tools that identify skill strengths and deficiencies, and they provide materials to teach missing skills so that standards can be mastered.

The starting point for attacking the student performance problem is understanding the purposes and functions of the three different types of assessments:

  • Formative assessments
  • Adaptive assessments
  • Diagnostic assessments

Formative/benchmark assessments measure whether an academic standard has been learned. Formative assessments are the most commonly used type of assessment. They are administered three or four times during the school year to provide data to teachers and schools about whether students and classes are mastering standards. These assessments measure whether a standard required for the current year’s curriculum—for example, eighth-grade math—has been learned. Some of the name brands of benchmark assessments are Acuity, ThinkLink, LearningStation, and Scantron Achievement Series. School systems value these assessments as determining whether students and classes have mastered the standards being measured on the end-of-year accountability test.

What formative/benchmark assessments do not do is measure whether students have or lack the prerequisite skills needed to master this year’s curriculum standard requirements. Given that more than 30% of students cannot master this year’s standards because they lack the necessary prerequisite skills, this shortcoming of formative/benchmark assessments—their failure to provide teachers with information about which prerequisite skills are missing and how to remediate them—is critical.

Adaptive assessments measure growth and identify where the student is on the learning-ladder continuum. A scale score, or RIT score, is provided to show at what level the student is currently performing. This type of assessment is the best measure of academic progress. While other assessments are like a line in the sand showing what a student has mastered, an adaptive assessment shows where a student is currently, where he or she has grown, and what level of instruction is needed to achieve grade-level performance. Adaptive assessments, like formative assessments, are administered two to four times a year. The name-brand adaptive assessments are NWEA Map and the Scantron PerformanceSeries. Unfortunately, adaptive assessments do not provide sufficient detail to specifically remediate deficiencies. After looking at results from adaptive assessments, teachers are usually left with the question, “Now what?”

Adaptive assessments are based on Item Response Theory (IRT). In IRT, students are given an initial question, and if they are successful, they will get a more difficult question until they have reached their top level of ability. If they miss a question, they will be given a lower-level question and will continue to descend until they reach their base level. This type of bracketing continues until the standard error of measurement meets a predefined level.

Diagnostic assessments determine missing skills and remediate them. Diagnostic assessments are the new kid on the block. The two name brands in this category are Knewton, a higher-education adaptive-learning and test-preparation product, and NWEA Skills Pointer. As Knewton has focused on the higher-education market, there is limited data on its impact in K-12 education, though the Knewton approach and algorithms are applicable to K-12 subjects. Knewton appears to include a large range of variables in its database, such as learning styles, as well as an algorithm similar to curriculum-based theory.

Skills Pointer is using Curriculum-Based Theory (CBT) to develop its assessments and remediation. Curriculum-Based Theory is framed around the learning objective. Questions are presented in a format similar to that used in adaptive assessments, but instead of using Item Response Theory, which moves the test score based on each question, CBT measures each skill. A learning ladder is prepared for each skill, and the student is tested with a sophisticated algorithm to determine if all the prerequisite skills have been mastered. This method allows a teacher to see a detailed learning path of all the skills a student needs to master to get back to grade level. The learning path includes a tutorial for the student and lesson plans for the teacher with practice activities for each missing skill. This approach solves the problem of teachers not knowing what to do after they have identified a deficiency. Like formative assessments, Skills Pointer focuses on mastery of standards, not growth or progress as do the adaptive assessments.

The main thesis of this month’s blog is to differentiate the three types of assessments: formative, adaptive, and diagnostic. All three have a place in every school, but they must be used for their individual intended purposes.

The increased use of diagnostic assessment instruments is necessary to solve the current issue of low performance on standards and to provide teachers with the tools they need to individualize instruction.

__________________________________________________________________________________________________

The BLEgroup is comprised of 100 leading K-12 education decision makers who focus on the integration of technology to improve educational practice. The BLEgroup works with both school systems and education technology firms. The monthly blogs are developed for both the industry and education decision makers by members of the group who have expertise in the topic.