The Art of Assessment - Guidelines for Scoring Open-Ended Responses
By now, many of us have heard the term “the Science of Reading” and while there is absolutely science behind the neural process of reading, and evidence-based instructional strategies, it must be noted that
Reading instruction is as much an art as a science.
And, we are going to go even further to state that the process of implementing data-driven instruction (very “sciency”) also has an art to it. Specifically, let’s talk about the art of assessment. Now assessment that drives instruction should absolutely be based on evidence-based practices. We talk frequently about the Literacy Processing Triangle. To read and write effectively, we need to make connections between phonological processing, orthographic processing, and semantic processing. Our assessments should be designed to measure a student’s ability in each of these areas.
It’s relatively easy to measure phonological and orthographic processing.
For phonological processing, we can look at things like:
Can the student blend syllables and sounds to create words?
Can the student segment syllables and sounds to support the spelling of words?
Can the student manipulate sounds in words by rhyming, isolating sounds, differentiating between similar sounds, and changing/adding/deleting a sound in a word to create a new word?
For orthographic processing, we can look at things like:
Can the student recognize visual differences between letters/words?
Does the student have age-appropriate sound-symbol correspondence (e.g., what is the sound “tch” makes?")?
Can the student read and spell age-appropriate words?
Assessing semantic processing is a little more difficult.
Semantic processing is more complicated to measure, so many assessments leave this out altogether. However, we must have data to understand how students are functioning in this area. These assessments, however, are typically a bit less straightforward than assessments looking at phonological and orthographic processing.
For semantic processing, we can look at things like:
Can the student understand vocabulary in context?
Can the student categorize words/concepts?
Can the student generate synonyms and antonyms for words?
Can the student comprehend oral or written information?
Assessing semantic processing is more difficult because these types of questions often have more than one correct answer. This means we need to use our judgment as educators to determine whether a student is “close enough” or got the gist of the question well enough to provide credit for the response.
Test developers often give an option of what a correct answer could be, but they generally can’t provide all possible correct answers because of factors such as space limitations on the page, regional differences in responses, and an ever-changing language (with new words seeming to appear daily) just to name a few.
All this said when assessing semantics,
There are a few guidelines we like to follow:
Query (or ask questions) when a student seems to be onto something but hasn’t fully articulated themselves in a way that proves to you (as the assessor) that they know the answer.
Be consistent about your administration across students (if one student gets credit for a specific response, all students get credit for that response) and also be consistent across time (if one response wasn’t counted correct in a baseline or pretest, it shouldn’t be counted correct in a final or follow-up test).
Provide credit for correct responses. This seems obvious, but it isn’t always. As mentioned before, there are many correct answers that don’t show up in an administration manual/instructor guide. In SMARTER Intervention assessments, we still recommend giving credit for any correct response. For example, if the question was “What is the opposite of big?” and the response options are “little, small” but the student says “mini” that is still an acceptable antonym for big, and therefore students should be provided credit.
Key takeaway -
Assessment is an art as much as a science, use your judgment for open-ended responses in whether or not a response should be accepted as correct!
If you’re looking for a comprehensive way to assess phonological awareness, phonics, vocabulary, fluency, and comprehension - check out our Delivering SMARTER Intervention program where we provide ready-to-administer assessments with both open-ended and multiple-choice response options for multiple student levels so you know exactly where to target your instruction!