One of the great mysteries of modern-day school reform is why we’re seeing such strong progress (in math at least, especially among our lowest-performing students) at the elementary and middle school levels, but not in high school.
Here’s Petrilli’s picture from Education Next.
Petrilli then asks:
Could it be that increased graduation rates are driving down twelfth-grade performance? Recent studies have indicated that graduation rates are up significantly over the past decade; that means that we have twelfth-graders in school today who previously would have dropped out. And those students are likely to be very low-achieving. Could they be pulling down the mean? Just like we see with the SAT as more students—and more lower-income students—take the exam?
I’m not a statistician but it seems plausible to me. Number-crunchers out there: What say ye?
Let’s begin by looking at the premise that there are a lot more students staying through 12th grade. Here’s a picture I made that shows the number of 12th grade students in public school as compared to the number in 8th grade four years earlier.
Now let’s do ye old number crunching. Turns out that if you combine my little chart above with the information Petrilli has already supplied then the answer is easy. The biggest effect of selection bias–that’s the term econometricians use for the kind of mixing of apples and oranges that Petrilli asks about–would be if all the increase in 12th graders came from students in the bottom of the score distribution. Since we have a 7 percentage point increase in enrollment we’d expect the lowest decile of scores to plummet. Look at the first graph. If anything, the bottom decile scores have improved slightly.
So there’s a clear answer to Petrilli’s pondery–nope. Too bad too; it would have been nice to find that the flat math test scores are just a statistical artifact.