From the Editor

More on Assessment

Anne Wujcik — Friday, February 17, 2012

Two interesting reports focused on assessments came to my attention this week.  ETS released an especially timely report on state pre-K assessment policies that examines the ways state-funded pre-K program across the country use assessments. At the other end of the spectrum, NWEA and Grunwald Associates released a report that looks at K-12 assessment from the varied perspectives of parents, teachers and district administrators. But first I want to point to a story in today's featured section. Maybe it's the frustrated history teacher in me, but I was delighted to learn that National History Day was awarded the 2011 National Humanities Medal, marking the first time that a K-12 education program received the prestigious award. The citation for National History Day was for being "a program that inspires in American students a passion for history." It does this by making history come to life, engaging students in an immediate, hands-on approach that involves them in rigorous historical research, careful analysis and interpretation of results. Students present their conclusions in a variety of ways ranging from formal papers to dramatizations and documentaries. More than more than half a million students participate in NHD projects each year. What I find so hopeful in all this is summed up in a comment made by author and journalist Cokie Roberts, a member of the NHD Honorary Cabinet. "History not only teaches students about the stories of our past, but is vital to creating a generation of young people who can apply these lessons to the future."

The NWEA/Grunwald report is full of interesting nuggets. As you would expect, what parents, teachers and administrators want from testing varies depending on role.  But there is a lot of agreement. The executive summary makes the point that two statements summarize the views of parents, teachers and district administrators on what they want K-12 education assessments (and could also inform the design specification of a new assessment system):

  • Zoom in for a close-up view of the performance, progress and needs of each individual child.
  • Zoom out by using a wider lens-from multiple angles, over many moments in time-to explore student achievement in a full range of subjects and skills.

Everyone would like to see more balance in the system - different types of assessments used at different times throughout the year; a broader (multiple subjects) and deeper (thinking and life skills) reach; better balance of time and money devoted to testing. I think balance is a real need, but as I read all the things that parents and educators want to see assessed - critical thinking and innovation, economics and art, media and civics literacy, to name just a few - it's clear that educational materials publishers and assessment providers have to get a lot smarter about how to gather reliable information about student performance across the spectrum without administering more and more tests.

Parents are most interested in in teaching and learning that is centered on their child. Teachers and administrators are most interested in individual student performance and personalized education, though administrators also are interested in measures of group performance and student growth over time. Parents are more concerned than educators with how well their children compare to other students within and outside of their districts.

Everyone also agrees that decisions about what students should be learning are best made at the local level. Teachers (50%) and parents (40%) say that decisions about what is being learned belong with classroom teachers, compared to 20% of district administrators. Over a third of district administrators believe these decisions should rest with district leaders, compared to 17% of teachers and parents who make this choice. Two interesting things here. Parents and teachers responses are closely match. And parents believe nearly as strongly as teachers do that decision about what students should be learning rests with classroom teachers. Clearly parents continue to trust and respect their children's teachers, despite all the press coverage and political posturing that too often portray teachers as the problem.

There's a lot more in the report, especially around the value usefulness and impact on teaching and learning of formative and interim assessments. The full report can be downloaded at http://grunwald.com/reports/index.php#nwea-report. You'll find a very useful InfoGraphic that summarizes key research findings and a short video there as well. 

The ETS report, State Pre-K Assessment Policies: Issues and Status, creates a useful baseline for practitioners and policy makers as more and more attention is focused on early childhood education, much of it coming from people who have limited understanding of the unique needs of young children and the ways in which early childhood programs differ from the classrooms that serve older students. If the way young children learn is different, then the ways that learning is assessed also needs to be different. The ETS report does a good job of discussing the special issues that should be considered when assessing young children and describing the three different approaches to documenting preschoolers' learning that are widely used in programs today. Those approached include:

Direct assessments - standardized, norm-referenced tests designed to provide both individual scores and aggregated data for large groups of children. These tests can provide evidence about whether children meet a specific skill or knowledge benchmark at a particular point in time.

Observation checklists and scales - teacher observations of children's classroom performance while engaged in everyday program activities. Teachers may record what they observe through the use of anecdotal or running records or they may use pre-set content checklists or scales that list indicators that demonstrate a predetermined standard of mastery for a discrete skill.

Samples of children's work - teachers collect purposeful samples of children's work. When combined with teacher-produced photos, notes, records, audio and/or video recordings, the resulting "portfolio" can provide a more complete picture of a child's skill set.

ETS researchers examined assessment policies used by 54 state pre-K programs in operation in 40 states and the District of Columbia. They found a preference for comprehensive observation-based protocols over direct assessments. Four programs require the use of measures that can be exclusively categorized as direct assessments, 19 programs require the use of measures that can be categorized as exclusively multi-domain, observational checklists or scales, eight programs report policies in which the measures cited represent a combination of direct assessments, observation protocols, and/or portfolios and 19 programs allow individual providers to choose one or all of the measures used to determine students' learning outcomes.

Of the 50 Pre-K programs that collect child outcome data, just three report policies requiring the annual administration and reporting of such data. Twenty-seven report that such measures must be administered and reported two or three times during the school year. Ten Pre-K programs have measure-dependent policies, typically with developmental screens administered once per year and observation protocols administered two times per year.
 
I think it's good news that these early childhood programs, unlike K-12 school systems, rely more heavily on multi-dimensional, observational tools than on direct measures (normative tests).  In fairness, in most cases standardized testing in public schools kicks in at the 3rd grade level, allowing more flexibility in K-2 classrooms, but standardized tests definitely rule. Pre-school programs are also more inclined to employ multiple measures.

It strikes me that preschools use of checklists and scales are akin to formative assessments in K-12 classrooms. The various observational measures or discrete skills can be aligned with the preschool curriculum or a program's early learning guidelines. Information gathered can then be used to inform instruction, helping educators make any adjustments that need to be made to the curriculum or to classroom activities.

The new Race to the Top - Early Learning Challenge grants ask recipients to focused on strengthening the use of assessments and their results to monitor individual children's progress, inform practice and improve program quality. And there is no question that there is room for improvement in many early childhood programs. But it's important that the steps taken to improve assessments both honor the traditions and practices of the best early childhood classrooms and focus clearly on improving practice and programs. The last thing we need is AYP for ECE.

Several weeks ago when writing about the Technology Assessments tool, I neglected to point readers to a useful assessment resource, Assess4ed. Managed by the State Educational Technology Directors Association, Assess4ed.net is an online community of practice. It assists states and districts in making the shift to online and computer-based student assessment, including implementing the Race to the Top Assessment program.  Assess4ed.net supports communication and collaboration between the private and public sectors, and – within states, districts, and schools –emphasizes the important roles for curriculum, assessment and technology staff necessary for implementation of the Common Core State Standards (CCSS) and their assessment.