Selecting Screening Tools
A quick story. I was working with a rural district here in Oregon with their MTSS implementation. They were at the stage of choosing an early literacy progress monitoring and screening system. I would visit the school once a month and do classroom walk-throughs and work side-by-side with the coach and principal. At the end of one visit they said, “Oh we’re meeting this afternoon with a sales rep from (this is a fake name) Fancy Schmancy Assessments, would you like to sit in and learn about the system?” So of course I stayed and watched the demonstration that was so colorful and had so many reports but also I noticed that the assessments were going to take about 20-30 minutes per kid in every classroom - which sounded much more like a diagnostic tool than a screener. Screeners are supposed to be short in duration. Towards the end of the meeting the principal asked me if I had any questions of Fancy Schmancy Assessments and I asked what I thought was a pretty basic question. I thought I would ask the question and they would just send me a pdf of the materials I was asking for and that would be it. So I asked the basics.
Key Questions to ask when selecting a universal screening assessment
Do you have a technical adequacy manual for your assessments?
Are the measures valid (which other measures are they positively correlated with)? Are the measures reliable (if two raters administer the same test will they get the same score)?
What are the characteristics of the sample of students that the assessments were normed on to develop the cut points for grade level and at-risk status?
The truth is I just asked the first question and the company’s rep asked back, “What’s a technical adequacy manual?”. Then I explained some of the info in the other questions. We left the meeting with a promise from the company rep to send a tech manual if they had one. Guess what? They didn’t! So the school instead used a set of measures that someone in the building already had training on and experience with and could help lead the implementation. Plus the assessment selected had a strong research base already established and was way cheaper! If your school or district is in this position please be sure to check reputable websites like the National Center on Intensive Interventions on their Academic Screening Tools Chart (linked here). While the list isn’t exhaustive you’ll have a much better understanding of the technical adequacy and rigor through the rating system. Plus you can compare assessments to each other. Just because a screening system is expensive and has lots of fancy colors and reports doesn’t mean that it has research or basic technical adequacy to back it up. It’s impossible to build a strong MTSS system on faulty meaningless data.
