If it’s accountable, local school staff will never grade their performance accurately

Kentucky’s Professional Growth and Effectiveness System (PGES) has problems

It’s inevitable human nature: If a person’s own organization is being held accountable for job performance scores, and if the organization’s own staff self-awards those scores, then those scores are going to get inflated to the point of meaninglessness.

A good example of this truth played out with Kentucky’s now defunct writing portfolios for school accountability program that ran under the also defunct KIRIS and CATS assessment programs. Teachers awarded students’ scores for the writing portfolios but the schools were then held accountable for those scores. As a consequence, serious inflation in portfolio scores was found by every single audit ever performed on the program.

But, Kentucky’s educators seem to have a slow learning curve. When the state’s Unbridled Learning accountability program started up, it was announced that elements would be added to the formula for the Program Reviews in subjects like Arts and Humanities, Practical Living and the operation of the writing programs in schools. Those Program Reviews were to be self-scored by staff in each school.

The inevitable result, as I wrote about in June, was that the first audit of the Program Reviews found notable inflation in scores. This was no surprise – it was just human nature.

At today’s Kentucky Board of Education meeting a discussion was held about obvious problems with yet another self-scored element that was to be added to the Unbridled Learning formula – the results from the teacher and principal performance evaluations conducted under the Professional Growth and Effectiveness System (PGES). To the board’s dismay, it turns out the vast majority of performance awards were in the two successful categories of “Accomplished” and “Exemplary.” A presentation of the score results showed that a whopping 93.5 percent of the teachers in the state got an overall job performance rating in one or other of the two success categories. Inflation for school leader job performance was equally bad with 89 percent scoring in one of the two performance success categories. To be honest, the scores are so unreasonably high that I don’t see how they can be useful in any way.

Those incredibly high school staff job rating scores didn’t sit well with school board members when it was pointed out that only around half of the state’s students were scoring proficient or better on the state’s KPREP reading and math tests. It was very quickly apparent to the board that adding the PGES scores into Unbridled Learning would instantly result in inflated overall school results that would undermine the credibility of the whole program.

The board wisely chose to reverse their earlier decision and removed the PGES from Unbridled Learning for now.

The really sad thing is that we really do need a viable school staff performance evaluation. I think some of the Kentucky Department of Education staff members have tried hard to create a viable program.

The problem is that I am unaware of any large scale school staff evaluation programs that ever worked well at the state level. So far, it seems teacher evaluations always get inflated even when the stakes are not very high. The only exception I am aware of involved a limited number of Kentucky’s Priority Schools where strongly independent auditors did the evaluations. This is a totally unaffordable approach for a statewide program, unfortunately.

The board also voted to reexamine the PGES situation next year, but I’m betting human nature will continue to mess this up just like it is inflating the Program Reviews that do count in Unbridled Learning (and inflate the results accordingly). If PGES goes into accountability, the results will just be inflated that much more.

For now, the most reliable part of Unbridled Learning involves the Next Generation Learners scores that were initially used by themselves when Unbridled Learning began in the 2011-12 school year. Next Generation Learners elements include things like test scores analyses and graduation rates. That data isn’t so easily corrupted by local schools (unless they outright cheat).

In any event, absent an unknown and unlikely change in human nature, I suspect in time most people will tend to disregard the overall Unbridled Learning results if those results continue to include inflated scoring elements like the Program Reviews. The focus will be on the Next Generation Learners scores. That focus won’t include everything we would like, but it will include only those things likely to result in stable and valid data.