How Kentucky’s blacks really compare to other states
That demographic disparity can give a very “white” state like Kentucky (still 84% white in our classrooms) a big advantage in simplistic comparisons of the scores.
Also, it is important to understand that the NAEP scores are only estimates. Like a pre-election survey of voters, NAEP results have a notable amount of sampling error, which can create inappropriate conclusions about real performance if state scores are just simplistically ranked as though they are highly accurate numbers.
So, using the NAEP Data Explorer’s statistical significance tools (as I and Stoneberg have suggested before), I ran some properly performed rankings of Kentucky’s black student performance compared to what is found in other states.
To very briefly summarize, our blacks have lost some ground compared to blacks in other states since the early days of KERA. The loss is more pronounced in the eighth grade.
To see the details, click the “Read more” link.
I assembled the following two tables of data for Kentuck’s black student’s performance on the very first State NAEP Mathematics Assessments and the new 2011 results. These tables take advantage of the NAEP Data Explorer’s ‘significance test’ tools to determine how many jurisdictions (the states plus Washington, DC) score statistically significantly higher, the same as, and statistically significantly lower than Kentucky’s blacks did.
The first table looks at all states that participated in each assessment. In the second table, I only look at data for states that consistently participated in both years listed.
The All States Results
This table includes all states that reported scores for blacks in both years. It is not truly “apples to apples” over time because not all states participated in the NAEP in the early years, and not all states have enough blacks to report scores for them even when they do participate. However, the data is still interesting.
KY Black State Comparison Oldest and 2011
Table Notes: Compiled from Main NAEP Data Explorer
In grade 4 testing in 1992, no state that reported results for blacks got a score that was statistically significantly higher than Kentucky’s blacks achieved. While I didn’t specifically check, given the listed standard errors (a measure of the plus and minus errors in the scores), I think Kentucky’s blacks scored statistically significantly higher than the national average in 1990. We tied 23 states and 11 states got scores statistically significantly lower.
Flash forward to 2011, and now four states outscore our blacks by a statistically significant amount, and 13 score lower. Our black’s math score is not significantly different from the national average, either.
Note that in 1992 only 35 jurisdictions reported black math scores for the fourth grade while in 2011 there were scores for 45 states.
Since things were different in 1992, I checked to see if the four states that outscored us in 2011 simply were not there in 1992. It turns out that Massachusetts, Texas and Maryland were present in 1990 and only tied us then. New Hampshire didn’t meet NAEP reporting requirements for black scores in 1990. So, three states definitely moved out ahead of our black students.
Any way you slice this, our fourth grade blacks lost some ground against three states and against the overall national average score, as well.
Also note that in 2011, once the haze created because NAEP is a statistically sampled assessment is considered, the best we can estimate is that Kentucky’s fourth grade black students could have ranked anywhere from 5th to 38th place for fourth grade math among the 50 states and Washington, DC. That’s not a very clear picture, is it?
The situation looks worse in the eighth grade. Of the 11 states that statistically significantly outscored us in 2011, six only tied us back in 1990, and one, North Carolina, scored statistically significantly lower than we did. The others didn’t have 1990 scores.
In 2011, our eighth grade blacks might rank anywhere from 12th place to 47th place for math. Again, that isn’t much solid information to hang our hats on, but the obvious and statistically significant decline in performance compared to seven other states since 1990 is very disturbing.
Results for states that consistently participated in both years
This is a fully compliant “apples to apples” comparison that only compares data for states that participated in both the early year assessment and the 2011 assessment and reported scores for blacks in both years. The downside to this approach is we have to completely ignore performance in states that didn’t have scores in the early years of State NAEP.
Consistent States KY Black State Comparison Oldest and 2011
As already mentioned, you can see that with this totally consistent group of NAEP participating states, blacks in none of them outscored our blacks in the early days of KERA. By 2011, blacks in three states statistically significantly outscored our blacks in fourth grade, and in eighth grade blacks in seven states surpassed ours.
In this consistent group of test participating states, only two got statistically significantly lower scores in grade 8 NAEP math than Kentucky received in 2011. That is a notable decay.
I’m not going to do a similar analysis of the reading scores because of Kentucky’s nation-leading exclusion of learning disabled students on the NAEP reading assessments in 2011 in both grades four and eight.
As Kentucky Commissioner of Education Terry Holliday pointed out in October, this creates serious questions about the validity of Kentucky’s reading scores, so the time and effort required simply isn’t worth it.