NAEP Versus the ACT College Entrance Test Reading Scores – Kentucky isn’t doing a good job of teaching reading

Arrival of the 2021 ACT Scores in a new format using an interactive web tool called the Data Visualization Tool got me thinking. How do Kentucky’s results from the National Assessment of Educational Progress (NAEP) from 2017, when the Class of 2021 was in eighth grade, compare to the class’s ACT results from 2021?

So, I extracted the ACT results from the “Percent Meeting Benchmarks” tab in the Data Visualization Tool and then turned to the online NAEP Data Explorer to extract the percentages that scored Proficient or Above on the 2017 NAEP Grade 8 Reading Assessment for several different Kentucky student groups.

The table below summarizes what I found.

Some key observations

As you can see, the agreement here is pretty close. None of the differences are even as large as five percentage points.

Thus, the consistent message about the Class of 2021 from both the ACT and the NAEP is, as we wrote last summer in “What Milton Wright knew about reading instruction, but lots of teachers apparently don’t,” that Kentucky’s schools are not doing nearly a good enough job in preparing our children in the most basic academic skill of all, the ability to read well. But, we could change that situation if we just followed through with some of the recommendations in the What Milton Wright Knew report.

More thoughts

The good agreement between NAEP and ACT reading results in the table indicates that the ACT and the NAEP are providing quite similar evaluations of Kentucky’s students’ reading abilities. So, despite the claims of some that the NAEP is simply scored too hard, the facts are that the NAEP provides a pretty useful indication about how well our students are being prepared, as of the eighth grade, for success in college – and even more.

You see, based on research from the ACT, the ACT results don’t just relate to college readiness anymore. Business and industry now say that the skills needed for success in college are also generally needed to succeed in non-college track living wage career jobs, as well.

This makes the agreement between NAEP and ACT even more compelling.

As a result, those often-heard comments that NAEP scoring is simply too hard don’t pan out in reality. At least for reading, it appears the NAEP is telling us the same story that the ACT provides, and that story is an accurate indication of the percentages of our students who are being well-prepared for the challenges of life.

I should point out that there are a few caveats to this comparison.

  • The Kentucky Department of Education hasn’t released public school only ACT Benchmark Scores for the class of 2021 and NAEP only shows scores for public school students. If the public school only ACT data were available, I would expect the results to be somewhat lower, especially for white students and for the average score for all students. That would likely further improve the agreement between ACT and NAEP for the average reading scores for all students and for the whites. It would slightly widen the differences for Black and Hispanic students, but probably not very much as fewer members of these racial groups are likely to be found in non-public school settings.

  • The NAEP scores are the result of a statistical sampling of students and have some plus or minus sampling errors associated with them. The true proficiency rates for all students might be slightly different from the results for those actually tested. In contrast, the ACT tested virtually the entire Kentucky high school graduating class of 2021 and any errors in these scores are likely to be small.

Still, the overall picture wouldn’t change much if these caveats could be overcome.

To sum up:

  • We would discount the accuracy of what the NAEP reading assessment proficiency rates tell us at our children’s peril. The scores appear to be quite valid estimates of preparation for college and careers in reading.

  • Evidence from multiple tests shows Kentucky isn’t doing anything close to an adequate job of teaching reading, and something is going to have to change.

  • All of which just adds more ammunition for parents who want to have more choices on where to send their child to school. If your school isn’t teaching your child to read, you should have the ability to try something else.