I am sure others can add their favorite examples of what the National Assessment of Educational Progress (NAEP) expects students to know by certain grades that NGSS ignores, but here is one good example from my area of interest as a graduate electrical engineer.

I found the problem in question while looking for questions from the new NAEP Technology and Engineering Literacy assessment. It is a fourth grade work problem from the 2009 NAEP Science Assessment. This work problem asks fourth grade students to explore some of the characteristics of electrical circuits using a battery, several light bulbs, and several samples of conductor and non-conductor materials.

The big problem: NAEP clearly expects fourth grade students to have some exposure to closed electric circuits. However, the Next Generation Science Standards (NGSS) all but ignore the subject and generally delays what bits and pieces regarding closed electrical circuits that (maybe) are covered until later grades.

There are several issues with this NAEP question.

One issue involves what is actually being measured. A good reader who has little prior knowledge of electrical circuits but can follow instructions could probably do quite well with this problem, though students who have already worked with these simple electrical devices in class would certainly have an edge. That would be especially true with the fairly challenging questions found with Diagram 6 in Part 3, which involve the concept of increasing resistance in a circuit (and for which I don't think the sample student answer to the question, "Explain what you think was happening in the circuit," is particularly good). Could this be more an assessment of reading ability or of science knowledge?

In any event, a key point is that the NAEP Science Framework for fourth grade students as of 2009 apparently expected those students to have some knowledge of closed electrical circuits.

Along come the Next Generation Science Standards (NGSS), and the term "electric circuit" is scarcely to be found. The one, lone occurrence is in a fourth grade example in the DCI Arrangement of the standards under standard 4-PS3-4's Clarification Statement, where it says:

"Apply scientific ideas to design, test, and refine a device that converts energy from one form to another.

* [Clarification Statement: Examples of devices could include electric circuits that convert electrical energy into motion energy of a vehicle, light, or sound; and, a passive solar heater that converts light into heat. Examples of constraints could include the materials, cost, or time to design the device.]"Notice that this talks about energy conversion but does not require discussion of the concept of closed electrical circuits. And, the use of the term "could" in the Clarification Statement clearly makes this optional.

Concepts introduced in the NAEP question, such as "electrical conductivity," don't get mentioned until higher grades ("electrical conductivity" isn't discussed until the fifth grade in NGSS, and then only as an optional example in the Clarifying Statement under 5-PS1-3 without any requirement to discuss closed electrical circuits). The term "wire" as in an electrical conductor does not show up in NGSS until middle school and then is only discussed in terms of the construction of an electromagnet but without any reference to closed electrical circuits. The term "battery" does not show up in the NGSS until high school.

So, electrical devices and concepts that the NAEP assumes students are introduced to no later than the fourth grade are options only for later grades or don't appear at all in the NGSS.

One last note: At the end of the NAEP problem, students are instructed to “put everything back into the large plastic bag.” That could create a hazard if the wires with alligator clips accidentally short out the battery. Some modern batteries can put out pretty decent amounts of power. Depending upon the size of the wire in the alligator clip assemblies, things might get pretty hot. At the very least, shorted batteries can leak corrosive chemicals, which isn’t a good thing, either.

This brings up another factor to be considered in the creation of higher-order assessment problems: unintended consequences.

Not only can such assessment items wind up not testing what is intended (e.g. is this mostly a reading problem or a real science problem?), but there can be other issues, even involving safety, too.

Go figure.