Back in October I wrote about Chicago’s “double-dose” algebra program for 9th grade students with below median math scores. Evidence from Kalena Cortes and Joshua Goodman showed that giving very weak students a second math class each day really helped them get better grades. Now the authors, joined by Takako Nomi, have some very interesting results about longer-run results. In fact, two interesting results on quite separate points.
The “standard” interesting result is, in the authors’ words, “substantial positive impacts of double-dose algebra on credits earned, test scores, high school graduation and college enrollment rates.” Here’s one of the authors’ pictures, which shows: (A) Students with math scores just low enough to qualify for double-dosing were notably more likely to graduate high school than were students with similar scores who just missed eligibility. (B) Poor readers benefited even more from math double-dosing than did students as a whole.
test score impacts of this policy dramatically understate its long-run benefits as measured by educational attainment …. In our sample, … estimates suggest that a 0.2 standard deviation increase in fall grade 11 math scores, the upper end of our estimated treatment effect, is associated with a 2 percentage point increase in college enrollment rates. We observe college enrollment effects roughly four times that size, highlighting the fact that long-run analyses of such interventions may yield very different conclusions than short-run analyses.
What’s going on? Test scores don’t measure everything. Test scores are often the best single measure of progress. “Best” doesn’t mean perfect. There’s a real research dilemma here. The authors demonstrate the importance of looking at non-test outcomes, such as college attendance. But looking at long-run outcomes means waiting for years or even decades for data to become available. Not much help for today’s policy makers.