Could inflate Kentucky’s College and Career Ready rates

Taking a responsible action, the Kentucky Department of Education has determined that the use of smart calculators on the COMPASS test from the ACT, Inc., could inflate scores and has now outlawed use of those calculators for testing.

The findings could impact the state’s reported College and Career Ready rates and could have national implications, as well.

This story originally broke about three weeks ago in a Herald-Leader report titled “Ky. orders review after questions raised about use of calculator software on state tests.”

As reported by the Herald-Leader’s Linda Blackford:

“At issue is the math portion of ACT's COMPASS test, which is used as a placement test for high school seniors who have not met college-readiness benchmarks on the ACT test they take as juniors.”

COMPASS is important in Kentucky. The results figure into Kentucky’s Unbridled Learning school accountability system. If a student fails to score high enough on the ACT college entrance test, he or she is counted as College and Career Ready by achieving a sufficiently high COMPASS score.

Kentucky has reported a dramatic increase in College and Career Ready rates in the past four years. At present it is uncertain how much of that improvement came from COMPASS results, but my own research shows that very little improvement has occurred in ACT performance over that period. Thus, the COMPASS testing issue now casts doubt on the Unbridled Learning claims of significant progress in this accountability area, a situation the department of education is right to take seriously.

The ACT rules allow students to take the COMPASS test only with certain types of calculators. Prohibited calculators include those with built-in algebra systems, but in a crazy quirk – which ACT is defending – some of the allowed calculators can be loaded with a software program called Zoom Math.

According to Professor Steve Newman, a math professor from Northern Kentucky University, students can solve algebraic equations using Zoom Math equipped calculators without understanding the math required. Newman conducted tests with some colleagues and discovered that using a Zoom Math equipped calculator could allow students to score well above the COMPASS passing score without really understanding the subject. The Kentucky Department of Education has now replicated those tests and finds there is a problem.

COMPASS is used nation-wide as a college placement test, so problems found in Kentucky could impact the futures of the 2.2 million students who use this test annually.

Interestingly, while ACT is defending its current policy for Zoom Math, the company that makes the software does not recommend using the program as a testing aid. The Herald-Leader’s original article says a number of Kentucky school districts have purchased the program and do allow it to be used on COMPASS. At least one district also allows Zoom Math to be used on the ACT college entrance test, as well.

Newman, who will be pleased with the new ruling, formerly said, “We cannot continue this fraud.”

At least one state already had banned such calculators on its tests. Mississippi banned the use of Zoom Math loaded calculators on its assessments after determining Zoom Math:

"Provided students with an unfair advantage and compromised our ability to make specific claims about student learning based on our assessment," according to spokeswoman Patrice Guilfoyle.