Back on November 1, 2013 I wrote about concerns with the first round of scores from Kentucky’s new Writing Program Reviews. I pointed to some extreme examples where the scores from the new reviews didn’t match well with other data related to student writing performance.

Apparently, our concerns are raising some awareness. On Friday, November 8, 2013 Kentucky Commissioner of Education Terry Holliday discussed in some detail how the first year results from the program reviews will be getting an extra look, including some auditing.

This is a good development, because it looks like the program review scores are all over the map. Some very high performing schools like Beechwood High School and Dupont Manual got the lowest Writing Program Review scores while some schools with chronic low performance got top scores.

Kentucky should not use program review scores for actual Unbridled Learning accountability before a careful review of quality is completed.

Furthermore, I still see a major problem. I suspect the department is going to relearn a lesson we had with every single audit that was ever performed of schools’ self-graded writing portfolios under the old KIRIS and CATS accountability systems. It just goes against basic human nature to expect schools to do this accurately when the accountability of the school relies on the results. When the dust settles, I think we are going to have to find a way for disinterested people from outside of the school district come in to do the program reviews. If this does not happen, the long history of inflated writing portfolio scores is almost certain to reappear, again.