Standardized tests all include measurement error. Never mind the hard conceptual questions of what we ought to be testing, standardized tests include error just because of the randomness of which question appears on a particular form and the like. Psychometricians measure the measurement error and the companies that sell the tests provide the measurement error measurements.
Koedel, Leatherman, and Parsons have figured out how to incorporate the published figures on measurement error into improved value-added measures. The intuition is that student test scores with lots of measurement error get less weight in calculating a teacher’s average value-added then do test scores with less measurement error. (Interestingly, tests are more accurate for students near the middle of the test score distribution than they are for students near either the top or the bottom.)
The authors argue that making their adjustment, which is pretty much costless (now that they’ve figured out how to do it), increases the accuracy of teacher evaluations significantly. As an example, they simulated sorting teachers into quality quintiles and found that the correction reduced the misclassification from 21 percent to 16 percent.
Score another victory for the thoughtful use of statistics.