Let’s rank Kentucky’s NAEP scores intelligently – Grade 8 Math — The Bluegrass Institute for Public Policy Solutions

Unfortunately, doing valid comparisons of Kentucky’s NAEP scores to those from other states isn’t so easy. For one thing, statistical sampling errors in NAEP turn many apparent “wins” into nothing more than ties. Other factors such as wildly different student demographics and the ever-present achievement gaps complicate state to state comparison of scores, as well. You can’t just compare overall “All Student” scores from the NAEP and get a valid idea about relative state education system performance. You have to break the data out and do more refined comparisons.

I have done that for the NAEP Grade 8 math results using the NAEP Data Explorer tool to generate maps that show states where white students scored statistically significantly higher, the same as, or statistically significantly lower than Kentucky on this math assessment series. The results are very disappointing.

This first map in Figure 1 covers the Grade 8 NAEP math results for 2015.

Figure 1

As you can see, after a quarter of a century of KERA and in the fourth year of testing to the Common Core State Standards, in 2015 Kentucky’s white eighth graders only statistically significantly outscored their counterparts in just two other states in the entire nation, West Virginia and Alabama. In turn, Kentucky’s white eighth graders were statistically significantly outscored by their counterparts in 42 other states and the District of Columbia school system.

There isn’t much change in 2015 from the results posted two years ago after the 2013 NAEP was published, as shown in Figure 2. We now outscore two states instead of just one in 2013, but the number of ties has gone down so that now one more “jurisdiction” outperforms us than we saw in 2013, as shown in the second map.

Figure 2

In 2013 Kentucky’s white eighth grade students only outscored their counterparts in just one other state, West Virginia. The Bluegrass State was outscored by 41 other states and the DC students in 2013.Now, take a look at Figure 3, which shows the situation for whites after the 2011 NAEP Grade 8 Math Assessment results came out.

Figure 3

In 2011 Kentucky’s white eighth graders outscored their counterparts in 3 other states and were in turn outscored by a statistically significant amount by whites in 39 states and the District of Columbia.

So, let’s summarize. As shown in Table 1, since 2011 Kentucky’s whites have consistently seen whites in more states outscore them by a statistically significant amount. In turn, between 2011 and 2015 Kentucky’s whites now outscore fewer whites than they did back in 2011.

Table 1

Clearly, this isn’t progress. When you consider that even as of 2015 NAEP testing, about four out of five Kentucky students were white, this becomes even more disturbing. The results shown in Table 1 impact the vast majority of Kentucky’s students. This is not great news, but Kentuckians who would ignore these facts do so at their children’s peril.

Another question that must be asked is whether Common Core’s math standards might be playing a role.

Back in late winter of 2011 when the NAEP was given, no state was really doing Common Core in the classroom, not even Kentucky. Kentucky had only voted to adopt the standards one year earlier. However, the tests being used as of the spring of 2011 were still the old CATS Kentucky Core Content Tests, so it is doubtful much emphasis was being placed on Common Core in the Bluegrass State’s classrooms at that time.

Kentucky introduced Common Core aligned math tests in the 2011-12 school year, a year earlier than any other state. By the time the 2015 NAEP came along, Kentucky was into the fourth year of using Common Core aligned tests and related curriculum. In contrast, a number of other states were just beginning Common Core testing in 2014-15. Could Common Core explain what is going on in Table 1?One more point: In 2013 I did an additional examination of how our poor, school lunch eligible whites did against poor whites elsewhere. I don’t feel comfortable doing that with the 2015 data, unfortunately. A huge problem has arisen with the national school lunch statistics in the NAEP because of a new program where any school that has at least 40 percent of the students eligible for lunch based on real need can now go into the “Community Eligibility Option” (CEO) lunch option and feed ALL of their students, even sons and daughters of wealthy parents. Even worse, this may impact the numbers reported with the NAEP test results in uneven ways across states. That has really messed up the validity of the NAEP school lunch data as a proxy for poverty. Absent a lot more information that so far I have not been able to locate, I won’t do work with the lunch data this year.

However, here is how Kentucky’s lunch eligible white students performed on the 2013 NAEP when the lunch data was more trustworthy.

Figure 4

Incredibly, in the last year of NAEP where lunch data has some value as a poverty indicator, Kentucky’s lunch eligible whites fared very poorly against their counterparts in other states. In fact, the map looks little different from the Figure 2, which shows results for all white students in each state.

Furthermore, when we look at the whites eligible for school lunch results from the 2011 NAEP Grade 8 Math Assessment, shown in Figure 5, it is clear that Kentucky’s poor whites lost a lot of ground between 2011 and 2013.

Figure 5

Table 2

G8 Math for Kentucky Vs Other States, Lunch Only

Summarizing the valid lunch data from the NAEP Grade 8 Math Assessments, Kentucky’s performance – even for its school lunch eligible students – clearly decayed between 2011 and 2013.So, any way you slice this, Kentucky’s middle school math performance does not look so great. In fact, you have to argue that compared to other states, our dominant school racial group, our white students, have actually lost ground since Common Core took over our classrooms.

That sure isn’t the picture the Prichard Committee/Kentucky Chamber report paints, but this is what has really happened once you start analyzing the data correctly.

By the way, I suspect some will try to use the Prichard/Chamber paper as an excuse for not needing to increase school choice in Kentucky. Clearly, the NAEP data above shows that is not a valid argument. In fact, the data above shows that even our white students very clearly need more options as their performance in math over the past four years has deteriorated compared to the counterparts elsewhere.

Technical Note: All the maps were assembled using the NAEP Data Explorer.