First thoughts on the new Kentucky Assessment and Accountability Results

The media seems to have done a fair job with their initial coverage of the scores, but there are troubling things as well that not all have picked up on.

For example, a US News and World Report painted a rather rosy picture, with an article titled “Kentucky Report Card Shows Some Improvement in Student Test Scores but Considerable Work Ahead.”

 I don’t think that captures the real picture very well.

While there were some areas of improvement, there also were areas where scores stayed flat or even declined from 2022 to 2023. They include:

Elementary School: editing and mechanics (a subset of the writing area),

Middle School: math and social studies, and, most troubling,

High School: reading, math, science and editing and mechanics.

More problematic is the implication in the USN&WR title about improvement. The Kentucky Summative Assessments (KSA) used in 2023 only were implemented for full-scale testing in 2022 (2021 KSA participation was decimated by COVID). As I discuss here, KSA results don’t really compare to earlier testing from 2019 and before when a different test series, the Kentucky Performance Rating for Educational Progress (KPREP), was in use.

Basically, implementation of the KSA destroyed the trend line back to the pre-COVID era.

So, we can’t get reliable information from the KSA about improvement relative to pre-pandemic performance.

The only Kentucky run testing that provides a trend line of testing information, and it’s only available for high schools, is from the ACT college entrance test. Kentucky has given the ACT to all high school juniors for more than a decade.

As you can see in the first table, which was extracted directly from the KDE’s Briefing Packet, State Release 2022-2023 Assessment and Accountability Results, back in 2018-19 the 11th grade ACT Composite Score was 19.0 and in 2022-23 it was only 18.5. This is up slightly from the 18.3 posted in 2022, but none of the listed scores for individual subjects or the overall Composite Score were as high in 2022-23 as they were before COVID.

And, overall, until we get more results from another test that has been rather stable through the COVID era, the National Assessment of Educational Progress, next due in 2024, getting a real handle on the COVID recovery in Kentucky is going to be sketchy, at best.

I have a few more thoughts about the new Accountability rating system in Kentucky. The 5-star system is gone, and now we have a 5-color system, instead.

And, the system is scored very differently, with all sorts of factors considered such as test scores, change in test scores from last year, school climate and safety, graduation rates (for high schools), and more. But, with so many variables added into the scoring formula, it is easy for important stuff to get watered down in the ratings.

This next table shows the 10 top-rated high schools in the new, 2022-23 scoring.

The second table first lists each school’s final score for the overall combined indicator rate along with the ranking for the school for that rate. I then show each school’s 2023 Grade 11 ACT Composite Score and the school’s ranking for that score. Next, I compare the proficiency rate for the state’s Kentucky Summative Assessment in Math and that metric’s ranking to the school’s score for ACT math and the ranking for that ACT academic area.

I think as you look through the various ranking entries, you will see that there is an interesting situation for a number of the top 10 schools. They might rank high using the scheme to mix a lot of stuff together to develop just one final number, but in the important area of academic performance, there is a problem.

For example, top-ranked Dawson Springs Jr./Sr. High School posted a rather weak ACT Composite score of 18.1, which ranked only in the 112th place, about half way down the list of Kentucky high schools.

Drop down a little further, and Green County High School did even worse on the ACT, with a 17.9 ACT Composite Score that ranked 124th.

Equally of concern, looking at the comparison of KSA math to ACT math, the rankings are also disturbingly different for some of these supposedly top of the stack high schools. Green County High ranked 19th for KSA math but was way down in 105th place for ACT math.

My initial impression is that the KSA math standard is notably lower than ACT’s. Perhaps something else is going on here, however, and discussion is definitely needed.

In fact, a good hard look at the state’s new accountability scoring system is clearly needed, as well. How did Dawson Springs Jr./Sr. High get such a high rating from Kentucky when its students clearly don’t measure up well on the ACT even just against students in other Kentucky public high schools?

Are we getting the right stuff, here? Someone needs to find out.