Too much 'politics' in some Kentucky legislators' 'understanding' of the state’s actual education performance

This past week has provided a number of disturbing examples recently that a number of state lawmakers legislators don’t have a good grasp on the data about Kentucky’s public school performance. The real picture is certainly different from what some pols would have us believe.

House Education Chairman Derrick Graham, D-Frankfort, sounded off in a Courier-Journal op-ed about his problems with Senate Bill 1, an education-reform measure sponsored by Sen. Mike Wilson, R-Bowling Green, who chairs his chamber’s education committee.

Regardless of that bill’s issues – and there are some – Graham made several comments about Kentucky’s education performance that are just flat wrong.

For instance, he claims:“…there is just too much positive data out there to imply the state hasn’t made remarkable progress in educating our children.”

Graham cites as supporting evidence this gem: “Kentucky elementary and middle school students have registered greater gains in reading and math on the National Assessment of Educational Progress in the past decade than students in almost all other states.”

Well, while our fourth-grade National Assessment of Educational Progress (NAEP) changes between 2005 and 2015 are high, the same cannot be said for Kentucky’s eighth-graders.

The table below shows the reported NAEP eighth grade scale scores for white students in all the states for reading and math for 2005 and 2015, the latest year of NAEP data available.

It’s broken out by race because if we only examine “all student” scores, Kentucky could get an unfair advantage because our state has a much higher percentage of whites in its classrooms than in the vast majority of states.

Couple high white enrollment with the massive racial achievement gaps that also occur on NAEP and something called “Simpson’s Paradox” tells us the “all student” scores can be misleading. (This is a topic I have discussed in the Bluegrass Policy Blog before.)

So, let’s see how our white eighth grade students compare to their counterparts in other states over the past decade on the NAEP:

All States NAEP G8 Reading and Math 2005 and 2015

Looking at the table, it’s clear that Kentucky’s eighth-grade students didn’t log greater gains in the last decade than “almost all other states.” In fact, our improvement has only been about average.

Even worse, as of 2015, our white eighth grade NAEP scores in both math and reading are statistically significantly lower than the national public school average scores for whites across the nation. When you are behind – as Kentucky is – it’s easier to make, and show, improvement.

Another NAEP graphic shows Kentucky’s NAEP proficiency rates for the most recent and earliest test years available. (Note: All NAEP results have statistical sampling errors, by the way, which are often shown with confidence interval markings. However, confidence intervals are not shown in this graph for purposes of simplicity. Such errors generally run somewhere around plus-or-minus two to three points for Kentucky’s proficiency rate scores.)

NAEP2

Even after a quarter-century of expensive education reform beginning with the Kentucky Education Reform Act of 1990 (KERA), NAEP proficiency rates for our commonwealth continue to lag far below acceptability.

Furthermore, based on rates of progress derived from the data in this graph, it’s generally going to be a very long time before we reach acceptable proficiency rates. For example, using current rates of progress in fourth-grade math – the subject area where Kentucky students show the fastest progress – we’re still more than three decades away from achieving a healthy 80-percent proficiency level.

In eighth-grade reading, our worst-performing area, it’s going to take far more than a century to hit an 80 percent proficiency rate if we continue at the current, very slow, rate of progress.

Has Kentucky made some improvement on NAEP over time. Absolutely! But at least for the eighth grade, is that improvement anything close to “remarkable?” Definitely not. And even our fourth-graders won’t post 80 percent proficiency rates in reading for another half-century.

Graham should have said this “regrettable” rather than “remarkable.”(Additional technical notes: Sharp readers might note that the changes shown in the table don’t always agree with the scores. For example, rounded to the nearest point, Nevada’s eighth grade reading scores were 261 in 2005 and 270 in 2015. That’s a difference of only 9 points although the table shows a difference of 10 points. In fact, the 10-point difference is what actually results after rounding the difference of the actual Nevada scores of 270.208323349597 for 2015 and 260.581456415572 for 2005, which are contained in the Excel spreadsheet I used to calculate the differences. Thus, the changes shown in the table are more accurate than those that sometimes result when calculations are conducted using the rounded scale score values. It also should be noted that all the data from NAEP have statistical sampling errors. While not shown in the table and graph above, those errors tend to further reduce differences between states, making Kentucky’s performance appear even more average and less impressive. All scale scores in the table and all the proficiency rates used to create the graph come from the NAEP Data Explorer Tool.)