Martin Carnoy and Richard Rothstein’s report released two weeks ago by the Economic Policy Institute, “What Do International Tests Really Show About U.S. Student Performance?” has received a good deal of press attention. That’s well deserved because it makes a wonderfully wonky point and does so in an understandable fashion. I fear that some of the lessons taken by the press may leap to a point that the authors didn’t make.
Here’s the backstory. When we compare test scores across countries, what we’re usually after is comparing how a student (given that student’s background) would fare in U.S. schools versus schools in another nation. We want an apples-to-apples comparison. Carnoy and Rothstein figured out that the 2009 PISA tests were administered to a lot more “low socioeconomic status” students in the U.S. than was true in other countries. That’s because the U.S. has more low SES students than other countries. Since these students score lower than do upper middle class students (in all countries), the average for the U.S. puts too much weight on low scoring students and artificially lowers the reported scores for the U.S. Pointing this out is good science, but both a science caveat and a press caveat are in order.
Science caveat: The authors measure socioeconomic status by the number of books in the home. The bottom group is children from homes with fewer than 10 books at home. The top group has more than 500. The idea is that students from homes with fewer than 10 books are comparable in the United States and South Korea. Are they? The number of books in the home is an awfully rough measure of social class, especially when comparing across countries.
Press caveat: The authors don’t say that American schools do as well as the competition. They say we aren’t so far behind as previous reports have shown. Here’s a quote
A re-estimated U.S. average PISA score…would…improve the U.S. place in the international ranking of all OECD countries, bringing the U.S. average score to fourth in reading and 10th in math. Conventional ranking…report that the U.S. average score is 14th in reading and 25th in math.
Wow. So it’s good news that we’re far behind rather than far, far behind?