Kentucky Tonight: Afterthoughts #2 – A Look at Disaggregated Data
I was honored to be a guest on Kentucky Tonight on January 28, 2013 to talk about proposals to raise the minimum high school dropout age in Kentucky to 18.
A number of interesting things came up during the discussions, and I have been doing some follow-on research as a consequence.
Around 13 minutes into the show I had a chance to introduce a graph like this one below. This shows the trend in high school graduation rates for 14 states plus Washington, DC where an Age 18 law has been in place for an extended period of time, thus providing a good set of trend lines.
I pointed out that the graph shows Age 18 isn’t working well in most of these states, with five states actually showing a decline in their graduation rates over time, and most of the rest not matching the overall national average annual rate of improvement, either.
My little study was immediately criticized during the show by Kentucky Representative Carl Rollins and Prichard Committee Chair Stu Silberman. Both pointed out that such an analysis needed to look deeper, examining different demographic characteristics in the different states.
Actually, that is a fair criticism. I just wish groups like the Prichard Committee would follow through on that with some of their own studies like their recent “Top 20 by 2020” update they posted in December, which only looks at overall NAEP scores for the states without breathing a word about how Kentucky excluded a ton of learning disabled kids from the reading assessments or that things do look a lot different once you disaggregate the data by race.
In any event, I did a little disaggregating of the graduation rate data from the National Center for Education Statistics, breaking out the graduation rates for both whites and blacks for the Age 18 education jurisdictions I’ve been following. This first graph shows how the 15 education jurisdictions performed for white graduation rates.
As you can see, the whites-only graph does not look much different from the graph where all students are averaged together. Whites in only five of the states had a graduation rate improvement that exceeded the overall US average rate of improvement, and five states actually saw declines in their white graduation rates even though they have Age 18 laws on their books.
By the way, Kentucky’s white-only graduation trend has been an annual rate of improvement of 0.55 percentage points per year, which is just slightly better than the overall US average rate of change. Only four Age 18 states did better than we did with our current Age 16 law.
Now, here is the blacks-only graph.
This actually looks somewhat worse for the Age 18 states. Seven of them show declines over time in their black student’s high school graduation rates. Only four states outperform the national black average, and two of those are virtual ties.
Kentucky’s blacks did notably better than almost every education jurisdiction shown on this slide, with an annual rate of improvement of 1.32 points per year. Only Oregon and Wisconsin topped us, and I have some reservations about Oregon due to the limited data available on that state’s black graduation rates (click the read more link if curious).
In any event, disaggregating the graduation rate data by race does not change the conclusion from my original graph. Age 18 does not have a good track record over time of helping improve high school completion.
Here is the table I used to generate the whites-only graph. I include data for Kentucky.
Notice that most states had good data availability all the way back to 2002-03.When a state had less than complete data, I ran a separate regression to determine the slope of the annual change with only those data points that were available for that state.
I am a little concerned about the rather thin data available for Oregon and the District of Columbia, however, and those states should be considered with caution when examining the whites graph. However, with DC at the very top of this table and Oregon at the other end, even if we deleted both of them, the overall message for the remaining 13 states would be the same. Age 18 isn’t working well for whites.
Now, here is the blacks-only table.
Once again, I have some concerns about the limited data for Oregon and the District of Columbia, but even with both states removed the overall results would still strongly challenge the notion that Age 18 laws, by themselves, are beneficial.
Now, both Representative Rollins and Mr. Silberman indicated they wanted to see disaggregation by things like parent education levels and marital status. I don’t have good sources for that data at present (Free and reduced cost lunch data for each state in the NAEP is restricted to lower grades only and may not reflect the high school situation very well, for example), and I don’t have time to look for that right now, but if anyone wants to suggest a reasonable data source that relates well to high school students, we can certainly look at those comparisons, too.
In any event, when you look at the 15 jurisdictions that have good trend lines with Age 18 dropout laws, there already seems to be a fair mix of north, south, east and west states. So, I doubt the picture is going to change very much regardless of how we try to disaggregate the data.
Technical Notes:
Source for 2002-03 to 2008-09 Averaged Freshman Graduation Rates by Race is an on line Excel spreadsheet from NCES.
Source for the 2009-10 Averaged Freshman Graduation Rates by Race is NCES Publication 2013-309.
The identification of states with Age 18 Laws in place since 2001 was discussed in an earlier blog found here.