More evidence Kentucky's larger than average high school graduation rates might not be a good thing
I’ve recently been writing about a highly problematic report from Johns Hopkins University titled “For All Kids, How Kentucky is Closing the High School Graduation Gap for Low-Income Students.” The Hopkins report has many problems, but the biggest issue is fundamental. The report assumes that diplomas awarded in different states require the same level of academic achievement. That is an unfortunate stretch. So, in this blog, I examine some limited academic evidence from the ACT for states that can be reasonably compared to each other. The results further undermine the Hopkins report’s major assumption.
Click the “Read more” link to see the full story.
I first commented about the Hopkins report’s major assumption in a blog about obvious problems with quality control of high school diploma awards in Kentucky. I point out in that blog that Hopkins makes a very questionable assumption that high school diplomas awarded in different states indicate similar levels of academic achievement. Given that there are no cross-state standards for awarding high school diplomas, that assumption is certainly dubious.
I amplify on problems with the Hopkins report in other blogs available here, here, and here. Those blogs concentrate on various technical issues with the Hopkins report, but the major underlying assumption about the quality of diplomas across the states always remains an issue.
The truth is that there seems to be little research, at least on line, regarding cross-state diploma equivalence issues. That’s no surprise because conducting such research across all 50 states is essentially impossible. No common test for high school students that reports scores by state is uniformly given to students in all 50 states.
Both the SAT and ACT are unsuitable. These college entrance tests are voluntary in most states; so, actual participation rates vary dramatically by state.
For example, in the 2013-14 school year, the ACT, Inc. reported that 100 percent of the high school graduates in only 12 states (including Kentucky) took this college entrance test at least once. In sharp contrast to Kentucky, only nine percent of Maine’s high school graduates in 2014 took the ACT. With such a low participation rate, and given the fact that the ACT tested graduates from Maine do not in any way represent a valid random sample of all Maine graduates, comparisons of ACT performance for school year 2013-14 between Kentucky and Maine are essentially meaningless.
The National Assessment of Educational Progress (NAEP) isn’t useful, either. The NAEP has never provided state-level results for 12th grade students across all 50 states. NAEP does provide information across states at both fourth and eighth grades, but that does not provide an adequate picture of what happens at the grade level where diploma decisions are made.
Other tests like the Armed Services Vocational Aptitude Battery (ASVAB) are focused on only certain skill sets and also do not provide valid random samples from each of the states, either.
So, there is no tool available to evaluate and compare the educational performance behind all 50 states’ high school diplomas.
Never the less, it is interesting to examine ACT results for the 12 states that did test all their graduates in the 2013-14 school year – the second school year of data considered in the Hopkins study. Since all grads tested in these states, there are no sampling issues to be worried about. There still are some limitations to this analysis, but it is interesting.
One limitation is that the available state-to-state scores from the ACT, Inc. include all students, public, private and homeschool combined.
In contrast, the 2014 graduation rates by state from the federal government only cover public school students. However, public school students predominate by a wide margin in all states, so it is reasonable to assume that the overall scores from ACT are fairly close to the scores for public school students only. That is certainly the case in Kentucky where the “all student” ACT Composite Score for 2014 high school graduates was 20.2 while the Kentucky public school only ACT Composite that year was 19.9.
Note: The Kentucky Department of Education has been far more aggressive about publishing the state’s public school only graduates’ ACT results than is true in at least some of the other states we will examine below. While our data doesn’t look so good, at least Kentuckians are getting better access to it.
Another limitation is one faithful readers of this blog understand. You cannot do meaningful comparisons of educational performances across states without breaking the data out by race. For new readers, an explanation of this requirement can be found here.
Now, with the foregoing discussed, we can start considering the table below, which compares Kentucky’s white students’ ACT performance and Four-Year Adjusted Cohort High School Graduation Rates for 2013-14 against whites in the 11 other states that also tested 100 percent of their graduates with the ACT in that school year. The left side of the table is sorted on the graduation rates. The right side is sorted on the ACT Composite Scores.
When we look at graduation rates, Kentucky ranks fairly high.
However, when we talk academic performance as reflected by ACT Composite Scores, Kentucky is tied for last place.
act-composite-compared-to-4-yr-acgr-for-100-pct-testing-states-in-2014
These results indicate that while the Bluegrass State does graduate proportionately more than most of the other listed states, the academic achievement those Kentucky graduates is lower. The table provides still more evidence that Kentucky’s high school diploma quality control is a problem.
So, rather than simply cheering a larger graduation rate – as folks at Johns Hopkins would like us to do – Kentuckians actually need to be concerned that too many of their students are receiving inadequate educations but that fact is being hidden by excessive social promotion to diplomas.
Here are a few more thoughts:
Notice that all three of the states that posted higher graduation rates than Kentucky – Tennessee, North Dakota and Illinois – also posted higher ACT Composite Score averages, as well. So, there was not as much social promotion in those three states as found in Kentucky. In fact, given its dramatically better ACT performance coupled with a larger high school graduation rate, Illinois offers an interesting example that higher standards for graduation plus more graduations can go hand-in-hand.
Why didn’t Johns Hopkins go looking for education answers in Illinois?
Perhaps someone else will check out what is happening in Illinois.
As far as we Kentuckians are concerned, the discussion above just adds more evidence that passing out more diplomas isn’t a solid indication that our kids are getting a better education. It looks more and more like Kentucky’s children, and the state’s adults, too, are just getting misled about our real education performance. “Research” from some folks isn’t helping us understand that.
Table data sources
The ACT Composite Score information comes from Table 1.5 in each individual state’s ACT Profile Report for 2014 (click on the state name here). This ACT data includes results for all white students, public, private and homeschool combined, but the overall score will be close to the public school numbers in each case because public school students greatly outnumber private school students in each state.
The graduation rates for white students come from a US Department of Education Excel spreadsheet titled “Provisional Data File: SY 2013-14 Four Year Regulatory Adjusted Cohort Graduation Rate, by State.”
Other references
Regarding Kentucky’s major problem with diploma quality control, we’ve written a lot. Here are some blogs and articles of interest:
Kentucky’s Effective High School Graduation Rate
Bluegrass Beacon: Social promotion enabled by watered-down diplomas
News release: The uneven quality of Kentucky’s high school diplomas
More on the quality control problems with Kentucky’s high school diplomas – Part 1
More on the quality control problems with Kentucky’s high school diplomas – Part 2
More on the quality control problems with Kentucky’s high school diplomas – Part 3
More on the quality control problems with Kentucky’s high school diplomas – Part 4