NAEP 2019 – How does Kentucky really compare to other states? Grade 4 Reading

One of the pitfalls of comparing state performances with the National Assessment of Educational Progress is ignoring the major differences in student demographics across different states by only comparing the overall average score for all students tested in each state. Doing such simplistic “all student” comparisons winds up comparing scores for lots of white students in Kentucky to scores for minorities who don’t do as well thanks to well-known achievement gaps. The picture that results is thus misleading.

Another pitfall is to ignore the fact that the NAEP is a sampled assessment and all the scores are subject to statistical sampling errors. It’s just like a poll taken before an election where the pollsters tell us that their anticipated voting statistics are accurate only to within a certain plus or minus percentage. Ignore the statistics in NAEP, and more misleading pictures emerge.

Fortunately, the NAEP Data Explorer web tool allows us to assemble some interesting comparison maps that can be broken out by race and also honor the sampling errors in the scores. If small differences are really just statistical ties, the Data Explorer’s maps will tell us that.

Using the NAEP Data Explorer’s map tools, I have assembled a series of maps that show how Kentucky stacks up against other states in both 2017 and 2019 for reading and math in both Grades 4 and Grade 8.

I’ll start with an area that has been one of the state’s strongest: NAEP Grade reading.

Figure 1 shows how Kentucky’s white students stacked up against the other states and the District of Columbia’s schools in 2017.

Figure 1

As you can see from the summary at the top only 16 states plus the DC schools (a total of 17 jurisdictions) had white student scores statistically significantly higher than Kentucky’s white student NAEP reading Scale Score of 227. That was statistically significantly lower than the overall white student score for public schools across the nation, as shown by the dark blue dot for the National Public score.

Thanks to those statistical sampling errors, Kentucky was in a tie with 31 states and statistically significantly outscored just 3 states.

You will see one circle for the Department of Defense Schools (DS) shown in gray. I didn’t include it in the actual calculations since the DOD schools are not really like a state or other political jurisdiction.

Now, let’s see what happened in the new, 2019 NAEP. Figure 2 tells that story.

Clearly, Kentucky’s white students lost ground in 2019 compared to whites in other states. NAEP Grade 4 reading results for 2019 now show 22 jurisdictions (21 states plus DC schools) statistically significantly outscored us. That’s an increase of five states definitely doing better than the Bluegrass State. At the other end of the scale, Kentucky only statistically significantly outscored just two states in 2019, down from the three states our white students bested in 2017.

Also note that our 2019 NAEP Scale Score for Grade reading also dropped by 2 points to 225. This was also below the National Public average, again, as well.

Overall, as of 2019 with 22 states doing statistically significantly better than Kentucky and only two doing statistically significantly worse, the idea that the Bluegrass State is performing at about the middle of the pack doesn’t look right. The state actually is floating around somewhere below that point for NAEP Grade 4 Reading for its white students, and with whites making up three out of four fourth grade students in 2019 in Kentucky’s public schools (according to the NAEP Data Explorer), that is something we all need to be concerned about.