2016 ACT again shows you have to disaggregate scores to compare state-to-state results

Before we get into comparisons of the data between Kentucky and elsewhere on the just-released 2016 high school graduates' ACT scores, I need to point out again why it's important to break the data out by race when you compare relative state-to-state performance in education, ACT scores most definitely included.

To explain why you have to look at the disaggregated scores, let’s look at the brand new, 2016 high school graduates’ ACT Composite Scores for Kentucky and charter-school rich Louisiana.

The table below summarizes the data. The data sources are the ACT, Inc.’s 2016 Profile Reports for Kentucky and Louisiana.

Kentucky Vs Louisiana 2016

Because these profile reports come directly from ACT, the data include graduates from all school types in each state – public, private and homeschool combined. However, the results probably match the public school results fairly closely.

First, look at the top line in the table, which shows the “All Students” data. This shows that there was a total of 50,809 high school graduates tested in Kentucky in 2016 and a fairly identical total of 48,692 graduates who tested in Louisiana. Other information found on Pages 14 and 15 in the “Condition of College & Career Readiness 2016, Nation” report shows that both states tested 100 percent of their graduates in 2016. This means comparisons between the two states are reasonable.

The table also reveals that Kentucky’s average ACT Composite Score for “all students” (actually all 2016 graduates) was 20.0 while Louisiana’s was lower at 19.5.So, Kentucky did better than Louisiana...Right?

WRONG!

Take a look at the rest of the rows in the table, which show what happens when we disaggregate the scores by race.

Very simply, Louisiana outscores Kentucky for every racial group except Asians, and the score difference for Asians really isn’t notable.

Now which state looks like it is performing better for its students?

This is why I object in this day and age about many reports we are still getting in Kentucky that only list overall “all students” scores. Those reports do not provide solid information about how Kentucky is really doing. In fact, those reports just inflate our state’s real educational performance.

We won’t make that mistake here, and we will now look at more state comparisons in our next blogs.

For those who like details, let’s break this out a bit more. Particularly for white students, which comprise each state’s predominant student population, Louisiana beats Kentucky by 0.5 point, enough to be notable on the 36-point ACT.

Even though Louisiana has a lot more blacks than Kentucky in both absolute numbers (15,550 versus 4,063) and as a percentage of its total graduates in 2016 (32 percent in Louisiana versus just 8 percent in Kentucky), the Pelican State’s blacks bested our blacks by a slight 0.1 point on the 2016 ACT.

For most of the other racial groups, Louisiana outperforms by about 0.5 point or so -- just as it did for its whites.

So, once again, the latest ACT data for Kentucky versus, dare I say it – charter-school-rich Louisiana – makes it clear, if you really want to compare performance on any testing program across states, you have to break things out by race. Otherwise, you wind up getting fooled by a statistical trap called Simpson’s Paradox. Simpson’s tells us that overall average scores can hide some really surprising information about what's really happening and it applies full force for Kentucky due to our state’s very non-standard student racial demographics.

While we have this table in front of you, note that in both states whites outscore blacks by a wide margin of 3.8 points for Kentucky and 4.2 points in Louisiana. Even though Kentucky’s whites tend to score low compared to whites in other states – including charter-school-rich Louisiana – Kentucky’s whites still outscore the vast majority of the racial minorities in other states by notable amounts. So, simply by having a higher proportion of whites than just about any other state, Kentucky’s overall score average will obscure a more accurate picture that can only be seen when scores are separately considered by race.

With these facts updated and refreshed in your mind, we’re ready to look at more from the 2016 ACT report in future blogs.(Minor edits for clarity August 24, 2016 at 12:03 pm)