There is a new challenge in determining how Kentucky’s NAEP results in 2022 stack up to other states. It has to do with a reduction in the sample sized NAEP used for testing. As I wrote earlier, this unfortunate change increases the statistical sampling errors in the 2022 scores, making a lot more results into statistical ties, sometimes when score differences are rather large.

Still, I thought I should update some of the comparisons in my Bluegrass Institute report about While Kentucky’s education system was sleeping …, so here goes.

Figure 1 shows how all the states ranked in 2019 and in 2022 for NAEP Grade 4 Reading Scale Scores for white students.

Figure 1

As you can see, Kentucky’s white students lost 5 Scale Score points between 2019 and 2022, and it fell a bit in the rankings accordingly. In 2019, 20 states statistically significantly outscored Kentucky, and in 2022 a total of 25 states statistically significantly outscored Kentucky. There is a clear decay in Kentucky’s performance relative to other states in these numbers.

It’s interesting to note that the NAEP Data Explorer web tool shows that in 2022, 74% of all Kentucky fourth grade students tested for NAEP were whites. Nationwide, only 45% of all public school students were white. In a total of 20 states, fewer than 50% of the students tested were white, as well.

Clearly, Figure 1 shows that for Kentucky’s dominant racial group, reading performance compared to other states declined even after the impact of the increased sampling errors is considered.

Now, let’s see how Kentucky’s white eighth graders performed compared to their racial peers elsewhere. Figure 2 shows the story.

Figure 2

In 2019, white students in 24 states scored statistically significantly higher than Kentucky’s whites and whites in two states scored statistically significantly lower.

Flash forward to 2022, and now whites in 23 states still score statistically significantly higher and only one state scores statistically significantly lower.

But, notice that in 2019 some states showed statistically significantly higher performance with scores only three or four points higher than Kentucky’s and one state was statistically significantly lower scoring with a score difference of just five points.

In 2022 we see two states scoring five points higher than Kentucky and one state scoring 5 points lower, but none of the differences are declared to be statistically significantly different. That is what happens when you do sampled testing with smaller samples. You lose precision.

Note, for example, Vermont scored three points above Kentucky in both 2019 and 2022. In 2019 the NAEP Data Explorer shows this was a statistically significant difference. In 2022, the NAEP Data Explorer says the difference was not statistically significantly different.

A more dramatic example is Nebraska. In 2019 it scored four points above Kentucky and that was good enough to be statistically significantly different. But, in 2022 Nebraska scored five points above Kentucky yet that wasn’t enough to be statistically significantly different.

Quite possibly, if the sample sizes had stayed the same as in 2019, even more states would have shown scores statistically significantly higher than Kentucky’s white eight graders achieved and perhaps one more state would have scored statistically significantly lower, as well.

Never the less, the information in Figures 1 and 2 shows Kentucky’s white students have not really made any reading progress of note since 2019 compared to their counterparts elsewhere and the Bluegrass State’s whites continue to score well below the middle of the pack in both grades.

Tech Note: The tables in Figures 1 and 2 were assembled from the statistical significance test feature in the NAEP Data Explorer.