Collapse of school accountability credibility in Kentucky – Again!
New Unbridled Learning school accountability results clearly inflated
Well, it looks like a third “Out” is richly deserved for the credibility of Kentucky education’s ability to self-grade its performance. Following the same, troubled trail of Kentucky’s former KIRIS and CATS public school accountability programs, Kentucky’s Unbridled Learning public school assessment and accountability system is now producing inflated pictures of performance.
How inflated are the school classifications in the 2016 Unbridled Learning report? Table 1 tells the story for high schools, by far the worst situation.
Table 1
high-schools-ul-classsifications-compared-to-individual-subject-p-rates-by-year
Back in 2012, the first year of Unbridled Learning school accountability, my analysis of data in the “Number of Schools and Districts by Classification” table on Page 5 in the Kentucky Department of Education’s Media Advisory 16-115 indicates that only 30.4 percent of the state’s high schools were classified as Proficient or Distinguished by the Unbridled Learning program. That classification percentage was lower than the percentages of students scoring at or above Proficient in five of the six academic subject areas reported and virtually tied with the proficiency rate for the sixth subject, science.
In 2012 I calculate that the average proficiency across all six academic subjects was 42.8 percent. That was notably higher than the overall percentage of high schools that reached the Proficient or better classification.
Now, look at the 2016 data. In 2016 an incredibly high 83.8 percent of our high schools received an overall classification of Proficient or Distinguished from Unbridled Learning.
However, the all-subject average proficiency rate was still way down in the 40-percent range at only 48.8 percent.
Still worse, in no case did individual subject area proficiency rates even begin to approach the overall classification figure of 83.8 percent proficient or more reported for 2016.That’s just not credible.
But, it gets worse.
The media advisory also shows how Kentucky’s Juniors performed on the ACT college entrance test in recent years after the ACT changed its score reporting to include some students who got extra time to take the test. Table 2 shows the percentages of students who scored high enough on the ACT to reach the Kentucky Council on Postsecondary Education’s (CPE) benchmark scores. Those CPE benchmarks allow students to avoid taking remedial courses in the related subjects as college freshmen at Kentucky’s public postsecondary institutions.
Table 2
ky-grade-11-act-cpe-benchmarks-2013-to-2016
As you can see, the benchmark results show there hasn’t been much change in college readiness over this four-year period. In fact, the math performance is essentially flat between 2012-13 and 2015-16. Furthermore, the math performance is notably lower than it was in 2013-14.CPE English benchmark performance also dropped between 2013-14 and 2015-16.Clearly, all of the benchmark percentages in Table 2 are far below the percentage of high schools rated proficient or more in the 2016 Unbridled Learning.
Referring back to Table 1, between 2013-14 and 2015-16, the percentage of Kentucky’s high schools that were classified as Proficient or above skyrocketed by more than 22 points. The CPE benchmark performance indicates no improvement anywhere near that magnitude occurred.
By the way, I’m not the only one concerned about the obvious inflation in the new Unbridled Learning results. Bluegrass Institute Scholar Gary Houchens, speaking on his own behalf and not for his college or the state board of education, says:
"The so called accountability system masks the real story of lackluster student achievement."
When Unbridled Learning says 83.8 percent of our high schools meet muster while far lower percentages of our students do, Dr. Houchens is obviously right.
By the way, Kentucky education’s self-scoring game started in 1992 with the introduction of the Kentucky Instructional Results Information System (KIRIS). KIRIS struck out in 1998 after becoming grossly inflated.
Kentucky education’s second “batter” in this game was the Commonwealth Accountability Testing System, the CATS as everyone called it. CATS struck out in 2009. Again, the strikeout call was based on obvious inflation in scores.
Now, the third batter in Kentucky public education’s attempt to field an accurate and credible school accountability program – the Unbridled Learning system – richly deserves being called out. Once again, inflation is now rampant in the Bluegrass State’s school classifications, especially so for the state’s high schools.
Clearly, the Unbridled Learning high school classifications for 2016 are nonsense.
So, here is a serious question going forward. Is it fair to our kids to expect the same people who run the school system to also judge its performance? Should evaluation of our public schools be conducted by a separate, neutral organization that doesn’t have to self-judge its performance with the sort of model we have been using without success since the early 1990s?We know the Kentucky Department of Education is hard at work on a replacement for Unbridled Learning, but is this the right agency to create that performance assessment system when a part of that performance is clearly the responsibility of the department itself?
Should legislators consider another model for school accountability, one where an independent evaluation agency conducts the assessment and accountability program?
Unfortunately, Kentucky has already struck out three times trying to get this right. The problem is that the ones who really lose in this game aren’t educators, they are our kids.
(Updated at 9:10 am to change "News Release" to "Media Advisory" and at 12:55 pm to correct data source pages in Table 1)