NAEP nonsense begins - Part 1

I knew the education crowd in Kentucky and elsewhere would not take long to jump on statistically insignificant score increases on the new National Assessment of Educational Progress (NAEP) math and reading results to puff up unsupportable claims of progress.

My first example out of the gate is the Kentucky Department of Education’s (KDE) news release.

The KDE’s news release TOTALLY ignores a lot of important facts of life about interpreting the NAEP scores, including the very basic fact that because NAEP is a sampled assessment, there are considerable amounts of plus and minus errors in the published scores. Those scores cannot be considered highly accurate, just like political polling prior to elections often turns out to have inaccuracies, too.

Anyway, let’s look at one specific KDE claim which was more or less repeated in a local Cincinnati area television news cast at 5 PM today:

• “Both 4th- and 8th-grade math scale scores in Kentucky improved from 2009 to 2011.”

Truth: As I showed in two earlier blog items today, Kentucky’s trends for both grades in both reading and math are FLAT from 2009 to 2011! The minor increases in eighth grade scores and fourth grade math (and the actual minor decrease in fourth grade reading) between 2009 and 2011 are ALL statistically insignificant. The KDE cannot use the NAEP to claim any improvement. It is statistical nonsense to do so.

The best NAEP can tell us about Kentucky’s education system is that its performance remained flat in math and reading in both fourth and eighth grade between 2009 and 2011. Anyone claiming differently does not know what they are talking about.