Are new Common Core tests providing inflated results?
A number of new state tests have come on line in the past four years as a result of adoption of the Common Core State Standards (CCSS). Several being used in more than one state include the ACT Aspire tests, the Partnership for Assessment of Readiness for College and Careers (PARCC) tests, and the Smarter Balanced Assessment Consortium (SBAC) tests. Other states have adopted their own, unique Common Core tests such as Kentucky’s KPREP tests.
A key concern about all of these new tests is whether or not they provide useful information about real readiness for college and/or careers. Today, we’ll take a quick look at several of those new tests. One is the ACT Aspire where we examine results from Alabama. The other is Kentucky’s “home grown” KPREP. We use the National Assessment of Educational Progress (NAEP) as the linking measurement for this analysis after presenting interesting evidence that the NAEP seems useful for this approach.
Key takeaways from this blog include:
• Grade 8 NAEP results from 2015 for Kentucky agree surprisingly well with the state’s ACT EXPLORE test scores. EXPLORE is a well-established college readiness test leveled for Grade 8 use. As in past years, Kentucky’s 2015 NAEP proficiency rates correlated rather closely to the state’s EXPLORE college readiness benchmark scores. This provides some evidence that the 2015 NAEP is useful to examine new Common Core aligned tests because Common Core supposedly is all about college and career readiness.
• In 2015 both the Kentucky-unique Common Core aligned KPREP tests and the Alabama results from the multi-state Common Core aligned Aspire tests show considerably higher proficiency rates than the NAEP reports. Given the close correlation of NAEP to Kentucky’s EXPLORE results and the much worse correlation of NAEP to both KPREP and Aspire, it appears both of these new Common Core aligned tests could be providing inflated indications of educational performance.
This first table shows the proficiency rates for Alabama’s students reported by the 2015 NAEP and the percentages of this same student group that were reported as proficient based on Aspire testing.
Alabama NAEP and Aspire 2015
As you can see, the percentages of students reported proficient on these two assessments in Alabama vary considerably, with the Aspire in every case reporting notably higher proficiency rates than NAEP returned. In addition, the amount of variation was considerable across the two subjects and two grades involved. There was only a 9-point difference in the Grade 4 Reading results, but the gap for Grade 4 Math was fairly considerable at 22 points.
Keep in mind that unlike Aspire, the NAEP only tests a sample of students in each state, so there is sampling error in all the NAEP scores. Data published with the scores in the NAEP Data Explorer indicate that the possible statistical sampling errors in Alabama’s NAEP scores probably run about plus or minus 3 points. Even so, all of the differences shown in the table above exceed that sampling error.
Some might ask, is NAEP a good gatekeeper for assessment accuracy. To help answer that question this next table presents the NAEP results for Kentucky’s Grade 8 testing in 2015. I compare that to Kentucky’s own KPREP Common Core aligned tests and to Kentucky’s EXPLORE test results for 2015.
Kentucky KPREP, NAEP and EXPLORE in 2015
EXPLORE is another test from ACT, but EXPLORE has been around for a while and has an established track record. The EXPLORE Benchmark Scores used to generate that test’s results in this second table are linked and equated to actual empirical results from the ACT college entrance test. That empirical study determined the ACT scores that translated into good odds of passing first year college courses.
As a note, Aspire has not been around long enough to establish such a track record.
First, notice that Kentucky’s KPREP Common Core aligned tests are returning notably higher proficiency rates than the NAEP is reporting, just as we saw in Alabama’s Common Core aligned Aspire results.
Now note that the differences between EXPLORE and NAEP in Kentucky’s 2015 testing were both quite close, within 4 points in both cases. In fact, once you consider that Kentucky’s NAEP scores also have sampling errors of about plus or minus 3 percentage points, the agreement of Kentucky EXPLORE to NAEP is astonishingly close.
Thus, it looks like both KPREP and Aspire are returning somewhat inflated results compared to the NAEP and EXPLORE. It also looks like agreement between EXPLORE and NAEP has been quite good, which makes it more likely that NAEP’s cut scores for proficiency in the eighth grade are set to good values.
Data Sources:
NAEP: Main NAEP Data Explorer Web Tool
EXPORE and KPREP: Kentucky School Report Card, State, 2015
Aspire: Reading
Aspire: Math
