Can a New Reading Assessment Tool Help Improve Literacy Rates?

In order to understand this article, long ago you went through the laborious (and heroic) process of learning to read. You began by mastering decoding skills—matching sounds to letters, and recognizing words. From there, you conquered comprehension skills, recognizing that words form sentences that create paragraphs like this one. This is an oversimplification, and there are many steps involved in each of these stages, but from a developmental perspective, reading is an iterative skill that builds on mastering previous skills and knowledge.

What if you had missed learning a fundamental skill along the way? And what if none of your teachers or caregivers recognized that gap before you moved on to the next school year? Reading, and learning itself, would become a source of frustration and heartache for you and those trying to teach you. If there were no intervention, your issues would compound—reading skills in early elementary school are predictive of a wide range of outcomes, including high school graduation, college attendance, socioeconomic status, likelihood of encountering the criminal justice system, and long-term health. 

In the U.S., approximately two thirds of students are reading below grade level, and the statistics are particularly distressing among Black, Hispanic, and low-income students. 

In addition, it’s estimated that about 10–15% of children have persistent struggles with reading due to dyslexia. How can we get teachers and administrators the tools they need to meaningfully improve the prospects of the more than 33 million kids who need help with reading?

The Rapid Online Assessment of Reading (ROAR), a project of the Stanford University Reading & Dyslexia Research Program and a recipient of Stage 2: Test Solutions funding from Stanford Impact Labs, is an online assessment platform and research project that aims to make reading assessment free, fun to take, and useful for teachers, administrators, and researchers. 

“Our overarching vision is to create a bridge between research and practice—as we work to develop more efficient and rigorously validated measures of reading development,” said Jason D. Yeatman, Associate Professor of Education and Pediatrics and the ROAR Program Director. “We want to make sure this research stays grounded in real-world problems faced by teachers and administrators and reflects the true diversity of learners at each stage of development. It’s a new model that bridges the lab, classroom, and community.”

Carrie Townley-Flores, Director of Research & Partnerships with ROAR, says, “Stanford Impact Labs funding has helped us tremendously in terms of scaling. We’re in a labor-intensive phase that requires simultaneous work on refining the technology and building relationships with partners so we can iterate the best tool possible and ultimately help as many kids as we can.” 

Improving and Expanding Assessment

ROAR improves on traditional reading assessment in a number of ways. It’s easy to administer, and it has validated assessments for years K–12. By offering a variety of test options, it provides more detailed and reliable results than other standardized assessments and the validation studies underlying the assessments are published in open-access, peer-reviewed scientific journals. Additionally, it’s free, which allows more schools to access the program and encourages flexibility with testing. For example, it makes it easy to test a middle school or high school student for foundational skills, which are typically not assessed after third grade. Students also report it is fun to engage with -- which goes a long way.

Ryland Adzich, project manager for ROAR at San Francisco City Academy (SFCA), a small private school of 60 kids who live in the city's notoriously under-resourced Tenderloin district, said that when SFCA administered the ROAR for the first time in the fall of 2023, students loved it. “The ROAR is set up like a video game, where a character like a monkey or a lion talks to you and guides you through the test. To the students, it felt like a fun, interactive game. This helped promote a positive testing environment where students could perform at their best.”

Adzich also added, “The test is also available in Spanish, and given that more than half our students are Hispanic, we had a handful take the ROAR in Spanish. This allowed us to see that these students don’t have trouble with reading skills, they need a different kind of support. We also tested our middle school kids and identified some areas of struggle that surprised us.” 

SFCA will be administering tests again, along with a new test on letters for younger kids, in the spring of 2024. Using the data from the two testing rounds, along with support from ROAR research coordinators, SFCA’s teachers and administrators will then decide how to implement interventions into the curriculum for the following school year. 

Creating a Virtuous Feedback Cycle

“Our research-practice partnership model brings collaborators into each stage of the research process and it has allowed us to bring innovations to stakeholders much faster than the typical laboratory research model,” said Yeatman. 

“Key to this process is sharing data with the schools and getting feedback from them on the tool so we can iterate and improve on what we’re offering,” said Townley-Flores. For example, the ROAR team recently created interactive score reports to be even more customizable and user friendly. “We got a lot of feedback from schools about what data would be useful and how best to visualize it. This feedback was critical to how we rebuilt the reporting system.”

Feedback to the lab has also resulted in the development of new assessment modules (e.g., letters, morphology), and is expanding the reach of the tool. Rebecca Sutherland, Associate Director of Research with Reading Reimagined, one of ROAR’s partners, explained, “ROAR is going to partner with an expert in Black American English to undertake a rigorous and innovative item analysis to reduce the tests' bias. Too often, standardized literacy assessments unreasonably penalize dialect speakers and underestimate their language and reading skills. This will be a game changer, showing the wider reading assessment field that linguistic bias reduction can and should be done.” 

Through this virtuous feedback cycle, ROAR is looking to continue to improve the tool with the goal of reaching 500,000 students within two years and seeing measurable improvement in reading proficiency, including reduced racial, ethnic, and economic achievement disparities in schools that use ROAR, within 5 to 10 years.