Research

Field of Education Moves Slowly

Theoretical improvements often take 30-50 years before wide-spread implementation. Often research-based practices are crowded out by the fad of the day, making the challenge of improving education even greater. Sub-disciplines within education far too often work in isolation. Simple solutions that do not address the complexity of individual students or the dynamics of a classroom at best have little impact and too often have a negative impact. Such has been the case in large-scale assessment where the use of assessment to drive curriculum and instruction has had numerous negative consequences.

What Have We Learned...

Coming to the University of Kansas gave me the opportunity to consider the fundamental issues of education from a broader perspective. As have many others before me, I had long realized that thinking about curriculum, instruction, and assessment needs to be integrated. However, few researchers attempted to develop models or theories to do this. I was impressed by the efforts of some, particularly the research trajectories of Susan Embretson and Kikumi Tatsuoka, but I remained frustrated regarding how incomplete this work was and how little impact it was having on federally mandated state assessment programs. This led me to develop three conference presentations in 2009 that served to focus my thinking. The first was presented at the National Council on Measurement in Education and was entitled, "What Have We Learned about the Structure of Learning from 30 Years of Research on Integrated Cognitive-Psychometric Models? Not Much." The second was presented at the American Educational Research Association Conference – The Efficacy of Formative Assessment: A Meta-Analysis. The third, presented at the National Conference on Student Assessment, was entitled, "Large-Scale Formative Assessment: Panacea, Transitional Tool, or Oxymoron."

6 Features Needed for a Large-Scale Assessment System

In 2010 an opportunity presented itself and allowed me to solidify my thinking. The US Department of Education issued a request for proposals to develop a large-scale assessment system for students with significant cognitive disabilities –the approximately one percent of students with the greatest learning challenges. It was clear to me that such an assessment system needed to do far more than measure learning – it needed to facilitate learning. I identified six features that needed to be present to do this. They are as follows:

  1. Comprehensive fine-grained learning maps that guide instruction and assessment
  2. A subset of particularly important nodes that serve as content standards to provide an organizational structure for teachers
  3. Instructionally embedded assessments that reinforce the primacy of instruction
  4. Instructionally relevant testlets that model good instruction and reinforce learning
  5. Accessibility by design
  6. Status and growth reporting that is readily actionable

Assessment that Supports the Needs of Learners

No one had ever tried to develop a learning environment in this way. Comprehensive fine-grained learning maps did not exist. The concept of instructionally relevant assessment previously was unnamed and in its infancy. Clearly much research – both basic and applied – was necessary and this has become the focus of my research. A closely related second area of research is assessment that supports the needs of learners who face educational or assessment challenges. This includes issues of test development and universal design and which have close ties to features 3-5 in the list above. I separate it as a research focus because it is also applicable to traditional testing programs.

  • Large-scale assessment
  • Computer-based testing
  • Diagnostic classification modelling
  • Learning maps
  • Test development
  • Score reporting
  • Assessment of students with significant cognitive disabilities
  • Assessment in higher education