More evidence that averaging test results can hide serious gap problems

As the Kentucky Board of Education works through trying to revise Kentucky’s public-school assessment and accountability system following the demise of Unbridled Learning, a new example from the Dunn Elementary School in Jefferson County shows that just averaging scores for groups of students can hide serious achievement gap problems.

Until now, the only gap accountability in Kentucky has been for a composite group of all special students averaged together. This composite group, known as the “Gap Group,” included all the racial minorities, learning disabled students, poor students eligible for school lunches and English language learners. That tended to bury problems for racial minorities.

In fact, a school could correctly claim it was making progress for its Gap Group while it continued to post white minus black achievement gap in math of more than 50 percentage points!

That’s just not right.

Our latest case in point of how averaging scores can hide serious problems: 2016-17 KPREP math test results for the Dunn Elementary School in Jefferson County.

Dunn first caught our attention almost two years ago when we published our Blacks Continue Falling Through Gaps in Louisville’s Schools, The 2016 Update report. The school had a district-leading white minus black math achievement gap of more than 50 percentage points in 2014-15 KPREP testing. Never the less, Dunn not only was not flagged for its minority student performance but also was identified as a top-performing Distinguished School in Unbridled Learning.

Flash forward to the present. While Unbridled Learning is dead, a month after the 2016-17 KPREP scores came out, Dunn’s School-Based Decision-Making Council (SBDM) was crowing about what looks like some nice improvements in their gap situation. The October 30, 2017 SBDM minutes for Dunn, which you can access online from links here, say the principal, “…reported the tremendous progress Dunn has had at clsoing (sic) the achievemnt (sic) gap from the 2016-17 KREP (sic) results.”

After you work through the typos, the minutes continue, saying,

“We increased the percent Proficient and Distinguished within our gap groups (sic):Increase of 4.8% in Reading P/DIncrease of 6.3% in Math P/DIncrease of 6.8% in Social Studies P/DIncrease of 7.2% in Writing on Demand P/D.”

Sure enough, if you access Dunn’s 2016-17 Kentucky School Report Card (access that from menus here) and go to the DELIVERY_TARGETS, PROFICIENCY/GAP tab and expand by clicking on the “Elementary School - All Students” links for each subject, Dunn did indeed increase the scores for its Gap Group exactly by the amounts the minutes claim (Just the one, composite Gap Group got the score changes shown in the minutes. This isn’t some sort of averaage for “groups”).But, there is more to the story.

First, recall that the Gap Group includes all students who fit into at least one of the following groups:

• Racial Minority

• Qualified for Free or Reduced Cost School Lunch

• Student with Learning Disabilities

• Student Learning English.

(See, for example, Slide 15 in this presentation).

Second, consider that Dunn has a notable number of Asian students, and they generally score even higher than the school’s whites. In 2016-17 KPREP math, for example, 100 percent of the school’s Asian students scored Proficient or more. Only 78.9 percent of the school’s whites matched that.

Third, realize that the LEARNING_ENVIRONMENT tab in Dunn’s report card says that the school has a total enrollment of 100 black students and 47 Asians. That proportion of high-scoring Asians is high enough to notably impact the overall “Gap Group” scores in ways that can hide what is happening to blacks in the school.

So, how does the white minus black achievement gap look at Dunn?

This table shows the white and black KPREP math scores and gaps in Dunn from the earliest to most recent KPREP administrations.

The scores in the table come from the ASSESSMENT_KPREP_LEVEL Excel Spreadsheets for listed year, which can be accessed from the Data Sets section of the Kentucky School Report Cards web site.

Below is graphed the white minus black gap trend from the table.

It is easy to see that the school’s white minus black math achievement gap has been bouncing up and down like a basketball moving down court, but Dunn’s white minus black math achievement gap has never been lower than 50 percentage points during the most recent four years.

In fact, the most recent, 2016-17 math gap for whites versus blacks in Dunn is worse than the gaps in 2011-12, 2012-13 and 2014-15.This isn’t progress. It is a sustained math gap that consistently ranks among the very worst throughout all Jefferson County elementary schools. But, a focus only on the Gap Group average performance hides what is happening to blacks in Dunn very effectively.

The Dunn SBDM minutes are silent about this continuing math gap, so it appears neither the school staff nor those on the school council know this problem exists. That’s the kind of thing that happens when you only look at scores averaged across a grouping of students.

Should the SBDM have looked deeper? Of course.

Should the state look deeper, as well? ABSOLUTELY! Because, as long as the state only pays attention to scores averaged across many students, some real problems are going to continue to fly below the radar.

It is true that the proposed accountability system for Kentucky will include both a Gap Group determination and gaps for major racial groups compared to the highest scoring racial group in each Kentucky school. But, the proposal will still average scores from the Gap Group with individual group gaps before making any accountability decisions. Thus, so the averaging problems so obvious in Unbridled Learning might not be remedied.

Meanwhile, blacks in Dunn continue to trail far behind the school’s whites in math. And, that just isn’t acceptable.