The results from the 2015 National Assessment of Educational Progress (NAEP) were released this morning. This is going to be a challenging set of data to evaluate due to a number of factors I will be discussing in future blogs as the various issues come up.

In any event, I know many are interested in at least an overview of how Kentucky performed over time. Fortunately, I was able to download the Excel spreadsheets that accompanied the NAEP release today in the “Data Quick View” area (scroll to bottom of web page to find). I also employed the NAEP Data Explorer to confirm an asterisk is incorrectly placed in one of the Data Quick View Excel files for the Grade 8 Math Scale Score for 2009 and to do some other quick checks.

Using these resources, I can make some initial observations about how Kentucky performed against itself over time.

Before doing that, I need to emphasize that all of the NAEP results contain statistical sampling errors. In fact, only a relatively small sample of students in each state is tested (For each individual grade and subject in 2015, Kentucky tested about six percent of its students on the NAEP). Those sampling errors often reduce what appear to be “wins” in different scores into nothing more than ties.

Unfortunately, a lot of NAEP analyses largely or completely ignore these pesky sampling errors. Too often, such reports wind up making claims the NAEP data really cannot support, often trying to turn ties into wins. I’ll try to avoid that mistake here.

Below I summarize Kentucky’s “All Student” NAEP Scale Score trends from the time just prior to the state adopting the Common Core State Standards to the present. Note in the tables that where an asterisk (*) appears after a score, that score is statistically significantly different from 2015 score. When no asterisk appears, the score is essentially just a tie with the 2015 score and no change in performance can be claimed.

With caveats out of the way, let’s take a quick look at Kentucky’s 2015 NAEP Scale Scores. Overall, the results are not impressive. Once we allow for that sampling error discussed above, our performance is flat compared to where we were way back in 2009 before Common Core came along.

Our biggest problem is in math, and it is more severe at the eighth grade.

As the asterisks next to the scores in the first table indicate, Kentucky’s 2015 NAEP Grade 8 Math Scale Score is statistically significantly lower than scores for both 2011 and 2013. The 2015 score is even one point lower than the 2009 score, though that difference isn’t significant once the sampling error is considered, so there is no asterisk for the 2009 score.

When it comes to eighth grade math, compared to 2009, which is the year before Kentucky adopted the Common Core State Standards, the NAEP shows Kentucky has made no discernable improvement (Note: the official NAEP Excel spreadsheet incorrectly shows an asterisk with the 2009 Kentucky score of 279. The NAEP Data Explorer and manual calculations confirm that is an error in the report).

Now let’s examine the Grade 4 NAEP Math results.

In Grade 4 math there is no statistically significant difference in any of the earlier scores compared to 2015. Even the three-point difference in the 2009 and 2015 scores is too small to overcome the sampling errors in the scores for these two years. Thus, there has been no detectable level of improvement in Kentucky’s Grade 4 NAEP Math performance since Common Core testing started in Kentucky in 2012.Kentucky does better in reading, so let’s check that out, too.

Kentucky’s NAEP 2015 Grade 4 Reading Scale Score is not statistically significantly different from scores in either 2009 or 2011. Essentially, despite some ups and downs in the scores, there is no difference in current performance than we saw back in 2009.Grade 8 reading presents a somewhat less encouraging picture.

The flatness in reading performance is more pronounced here. Kentucky’s 2015 NAEP Grade 8 Reading Scale Score is not statistically significantly different from any previous score back to 2009. In fact, the 2015 score is actually a statistically insignificant amount lower than the scores for both 2011 and 2013.As a note, changing student demographics tend to depress NAEP results everywhere in the US. As a quick check on this, I learned from the NAEP Data Explorer that the percentage of whites in the NAEP samples for Grade 4 math in Kentucky changed from 83 percent in 2009 to 78 percent in 2015. This is about an average amount of change for the states and probably isn’t large enough to impact the comparison of 2009 to 2015 scores to a major degree. In 2015 Kentucky remained near the top of all the states for its percentage of whites in public schools, ranking in a tie for 5th place.

Overall conclusions:

• The NAEP "All Student" Scale Scores indicate there has been no statistically significant improvement in Kentucky’s performance in either reading or math in either the fourth or the eighth grade since the year before Kentucky adopted the Common Core State Standards.

• If Common Core had impacts in Kentucky, they are too small for the NAEP to confidently detect them.

• Given that Kentucky has more experience with Common Core than any other state – including more years of state Common Core testing, which started back in the 2011-12 school year – the state’s flat performance in “All Student” scores is of concern well beyond the Bluegrass State’s borders.

I still have a ton of data to examine, and I will be looking at “takes” from others, too. So, stay tuned.