Information about charter school performance from the National Assessment of Educational Progress (NAEP) is sketchy and inconclusive. An important part of the problem is that the NAEP student sample sizes for charter schools have been rather small, which creates a lot of measurement error in NAEP charter school scores. The large measurement error in turn makes it impossible to detect small to modest trends.

Still, given that the trends shown below may not yet have risen to the point where they can be declared statistically significant, there does seem to be a trend in the nationwide NAEP Grade 4 mathematics results for students of different races who are eligible for the federal free and reduced cost lunch program (a proxy for poverty). The trend favors charter schools.

Here is how the NAEP results for white students eligible for the school lunch program look according to the NAEP Data Explorer.

Notice that in 2003 (earliest charter data collected in NAEP), charter school poor whites scored a point behind poor whites in traditional schools across the nation. As of the latest data for 2011, that has changed, and poor whites in charter schools now outscore their public school counterparts by 4 points.

Also note that between 2003 and 2011 poor whites in charter schools improved their scores for fourth grade math by 11 points, while their public school counterparts only gained 6 points.

Now look at the data for blacks.

The score differences for blacks have shifted around a bit, which may be due to the inadequate sampling sizes in charter schools, but over time the trend does seem to favor charter school students in the lunch program. I ran a regression of the change in scores over time, and there is a positive slope to the best fit line. That supports a trend favoring blacks in charter schools.

Lunch eligible charter school blacks improved their score by 12 points while blacks in traditional public schools only improved by 8 points.

Finally, here are the results for Hispanics in the lunch program.

The trend in the difference scores is the most dramatic of all, with lunch eligible Hispanics in charter schools moving from two points behind to six points ahead of their traditional school counterparts.

Lunch eligible charter Hispanics increased their scores from 216 to 232 between 2003 and 2011, a 16-point rise. Their public school counterparts only improved by 8 points.

I need to point out some caveats to this data. There is no information on exclusion rates for learning disabled and English language learners broken out by charter and non-charter schools. Different levels of exclusion could bias the data.

Also, it is reported that a lot of poor students in charter schools don’t join the federal lunch program simply because the charter schools they attend do not offer it. Those kids are still poor, but they don’t show up that way in the data. That could bias the charter school information, as well.

These caveats further highlight the limits of the NAEP data, of course. Hopefully, NAEP’s collection of charter school performance will improve in the future.