Kentucky education’s Program Review problems – We told you so – Now even Prichard sees the problem!

A few days ago, the Herald-Leader caught up to us with their story about “State audit report says schools are rating themselves too high in program reviews.”

At issue: the unavoidable fact that the Kentucky Department of Education’s new Unbridled Learning Program Reviews are suffering from massive score inflation – something that’s just about inevitable as we already pointed out here, here, and here, to cite just a few more recent examples (use the search feature in this blog with “Program Review” to find more).

This problem exists because school staff members grade their own programs and then those grades are used for part of the school’s Unbridled Learning accountability score.

Human nature inevitably takes over, and scores go through the roof.

Currently, schools conduct Program Reviews for their kindergarten through third grade program, for their writing program, for the way they teach practical living/career studies, and for the arts and humanities program. A fifth review, for World Languages, will soon be added to the mix.

The Herald-Leader says department of education folks hope doing more audits will help improve the Program Reviews, but the sad reality is that Kentucky has a lot of experience with schools self-evaluating parts of the state’s assessment and accountability program. Audits never helped enough.

This long trail of evidence that human nature trumps audits comes from the writing portfolios for assessment part of Kentucky’s now defunct KIRIS and CATS school accountability programs. Even though some progress was made over time, far too many writing portfolio scores remained inflated in the last years of the portfolios for assessment program.

Furthermore, the writing portfolio audit program was far more extensive than the proposed Program Review audit program. In the 2006-07 school term, a department audit showed 294 schools were involved with the writing portfolio audits. In the last year of the writing portfolio audits in 2007-08, that year’s audit report showed a total of 299 schools were included (Sorry, neither audit remains online). That last writing portfolios in assessment audit included more than ten times the number of schools that will get Program Review audits and this still wasn’t enough. Thus, human nature has already trumped a far more extensive audit program than the one the department is planning for this year.

Even Susan Weston, a key blogger for the Prichard Committee, told the Herald-Leader that the findings present a "quite serious problem." Weston continued, saying, “It's quite a big deal because we currently don't have a system that is working."

Kentucky Board of Education Chair Roger Marcum also told the Herald-Leader that “he was concerned about whether program reviews provided an accurate reflection of the quality of programs.” No kidding! Obviously, inflated Program Reviews don’t help our teachers improve and don’t help get our kids better educations, either.

In the meantime, the results from Unbridled Learning have rather evident inflation in a key element that counted for 23 percent of the overall Unbridled Learning score in 2014-15 (See definition for “Program Review Total Points” here). The program reviews are scheduled to count for 23 percent of the overall Unbridled Learning scores for 2015-16, too.

Parents, students, legislators and taxpayers beware.