Beware Kentucky School Report Cards Errors

I was doing some research in the Kentucky School Report Cards web site this weekend when I discovered a really misleading graphing error in the presentation of KPREP test scores broken out by race for schools and districts.

Here’s how I found this. You click on the link for the site, and start entering a school or district name. As you type one or more school names will start appearing. When the full school name you want appears, click on that to find the data you want.

I started out by entering “Iroquois High”. So far, so good. That brought up Iroquois High School’s report card home page.

But, I wanted some detailed proficiency rate data. So, I clicked on the “Proficiency Dashboard” link on the left side of the page. I then clicked on the “Academic Performance by Race” link at the top of the new page that opened.

That produced Big Problem 1.

Although I started out in the Iroquois High’s home page, when I got to the Academic Performance by Race page, the report card system switches – without warning – to Adair County and the Adair County Elementary School.

The district and school names are shown in pretty small type compared to other entries on the page, so you could easily miss the fact that a switch has been made. It is easy to wind up thinking the data is for the school you wanted when it isn’t the right data at all.

But, there is more to the story.

Once you reload the “Academic Performance by Race” page with the proper district name and the proper school name, the top portion of the page that comes up is as shown in Figure 1.

Figure 1

We are now looking at the correct data for Iroquois High School, but there’s yet another sneaky problem.

Although we are interested in the high school’s data, note the first two horizontal sections show in the web page are for elementary and middle school results. Results for the district are shown for these two school levels, but the right side of the web page where the school data appears is blank for elementary and middle school information because Iroquois obviously doesn’t have any results for those lower school levels.

But, look at the bottom horizontal section where the high school data appears. Note how the bars for the Iroquois data are longer than the similar bars for the district data. So, the high school appears to be performing above the district averages, right? WRONG!

Check the dark blue line at the very bottom on the left. It shows the proficiency rate for reading for white students. District-wide in Jefferson County, the white student proficiency rate for reading is 52.6%. Now, look at the dark blue line on the right, which supposedly represents Iroquois High School’s reading proficiency rate. The line is clearly longer than the line on the left, but the actual white student proficiency rate in Iroquois High School is only 9.7%, MUCH lower than the district overall average!

Exactly the same type of misleading information is shown by the bars for African-American and Hispanic students. The actual district-wide reading proficiency rates are MUCH higher than Iroquois High’s but the bar graphs tell the opposite story in both cases.

But, the mistake isn’t uniform. Asian students in Iroquois had a zero percent reading proficiency rate, and in this one case the bar, which is really just a line and easy to miss, is correct compared to the bar for the district-wide Asian reading proficiency rate.

A casual reader might never catch all of this and get mislead by the non-standard bars into thinking that Iroquois is doing much better than, not much worse than, the overall district for most of its different racial groups. Or, a really casual reader might wind up looking at the data for Adair's elementary school and never know what the real situation is.

I wondered if this was just a fluke, so I repeated the process to obtain data for Jefferson County’s Southern High School. Figure 2 shows what showed up.

Figure 2

Here we go again. Southern’s white student reading proficiency rate is actually only slightly more than half of the overall district average rate, but anyone just looking quickly at the bar graphs would never know that. Southern’s bars for whites, African-Americans and Hispanics, just like Iroquois’ bars for those racial groups, are longer than the district’s bars.

Go figure!

So, read those report cards carefully! And watch out for unannounced switcheroos in district and school names if you dig into lower levels of the report cards!

Oh, there is $till more on the report card$ to come, so $tay tuned.