Do we have to fix neighborhoods to get better education for poor kids? Let’s hope not. There’s strong evidence that better neighborhoods don’t improve educational outcomes if the better neighborhoods don’t also have better schools. Roland Fryer and Lawrence Katz write about the evidence in a paper in the newest American Economic Review Papers & Proceedings. (Katz also talks about the evidence in his Society of Labor Economics Presidential address.)
A federal government experiment between 1994 and 1998 (“Moving to Opportunity”) randomly chose poor families living in public housing in high-poverty neighborhoods to receive a housing voucher that let them move to much better neighborhoods. Fryer and Katz follow-up on the differences the vouchers made after a decade or more. Those who moved did end up in much lower poverty neighborhoods, poverty rates dropped roughly from 53 percent to 34 percent. There were good results from the move in terms of health and self-assessed well-being. (See Ludwig et. al.) So in important ways the better neighborhoods improved lives of poor people.
But schools weren’t much different in the new neighborhoods. Average test scores in the new schools were only three percentile points higher and free or reduced-price lunch participation fell by only four percentage points. So what happened to education with better neighborhoods but the same-old kind of schools?
Academic achievement? Educational attainment? Essentially zip. Nothing improved. And this includes kids who grew up in the better neighborhoods, not just those who moved after starting school.
Very discouraging news for those who think that if only we can fix neighborhoods our educational problems will be taken care of too.
The draft accreditation standards for schools of education call for higher average high school GPAs for teacher training programs, specifically that the average GPA should be 3.0 or better. Will raising this standard have a disparate impact on minority students? I think not.
Here’s a table I made from Baccalaureate and Beyond that breaks down high school GPA’s by race.
Note that the majority of students from all groups already have GPAs above 3.0. So there’s not too much reason to be concerned that a GPA requirement will affect any group very much.
One caveat: These are national numbers. I don’t know if there are particular teacher training programs with a clientele centered on minority students with low GPAs.
Economists largely believe that simply increasing spending on schools doesn’t do much to improve students’ outcomes. An important part of the evidence is due to “David Card and A. Abigail Payne”.
Card and Payne looked at court ordered changes in equalization payments. They found that moving money into poor districts (probably) helped close the gap between student outcomes in poor versus rich districts, but that the effect was small.
You can’t just compare student outcomes in low spending versus high spending districts because factors other than spending also differ. Card and Payne focused on changes that were exogenous. The effect of court orders was to move money toward poor districts, presumably with very little change in who the districts were educating.
spending equalizations that followed …court rulings in 12 states over the 1980s closed the gap in average SAT scores between children with highly-educated and poorly-educated parents by about 8 points, or roughly 5 percent.
Five percent isn’t much.
Highly recommended reading: “What Can Be Done To Improve Struggling High Schools?”. Also very easy reading. Also very depressing reading.
Julie Berry Cullen and coauthors note that almost all high schools emphasize preparing students for an academic/college bound track, and put together a case that for the many students who enter high school far behind this policy simply fails. They argue that smart vocational training would help such students move part way up the economic ladder, while the single-minded academic emphasis gets them nothing at all.
The authors understand perfectly well what a distressing thing this is to hear. I recommend the article because (a) they might be right (I hope not, and suspect they hope not too); (b) they give very straightforward summaries of what we know about interventions that haven’t worked; and (c) they describe some interventions that do seem to hold promise.
Among the interventions examined are:
- Just throwing in money (doesn’t work).
- Better principals and teachers (may well work, but hard to arrange).
- Allowing students to transfer to better schools. (Students don’t often take up the opportunity.)
- “Whole school reform” (some positive effect).
- Small schools movement (some limited successes, but largely given up on).
- Career and technical magnet schools (some evidence of success).
- Charter schools (no better on average than other schools).
- “No excuses” schools (evidence of significant success, but hard to scale up).
Here are a few taste-whetting quotes from the article:
…we suggest that underperforming high schools are failing in large part because traditional paradigms do not meet the needs of many of their students. The majority of high schools have sought to provide all students with academic skills from a primarily college-preparatory and nonexperiential perspective, with limited nonacademic supports. … Yet, this emphasis is likely setting many high schools up to fight a losing battle, because a higher proportion of students from disadvantaged backgrounds lack the requisite skills to succeed by this definition.
In essence, our advice to high schools when it comes to underperforming students is to redefine the mission and eschew traditional success metrics like test scores, focusing instead on more pragmatic objectives like keeping kids out of trouble, giving them practical life skills, and helping with labor market integration. That conclusion will no doubt be unsatisfying to many readers. In an ideal world, high schools would perform miracles, bringing struggling students back from the brink and launching them towards four-year college degrees. Indeed, a few remarkable and innovative schools seem to be succeeding at that lofty objective.
Focusing policy on changes in resources may be effective in early grades, where there is potential for such investments to improve students’ cognitive and noncognitive skill levels and trajectories. In contrast, the area of reform with the largest potential to improve high school outcomes like graduation is to provide struggling students with an increased variety of targeted educational models and schools….The most hopeful results have been seen in this area.
The draft accreditation standards for schools of education call for higher average SAT scores for teacher training programs. This raises an uncomfortable question about race. Students from racial minorities score lower on the SAT than do white students. Will the new standards reduce the number of minority students who become teachers? There’s not a definitive answer, but there’s good reason to worry.
The draft accreditation standards don’t have an SAT cutoff, but my back of the envelope calculations suggest that ed schools will largely be recruiting from students with SAT’s above 880 (old scale). I made a table from Baccalaureate and Beyond that breaks down SAT’s by race.
Only about two-thirds as many black students score above 880 when compared to white students. For Hispanic students the number is closer to a half. So there is real cause for concern.
But a reminder: The new accreditation standards set an average SAT goal. They don’t set a cutoff that applies to any particular student. Ed schools are free to recruit low SAT students who have other good characteristics, so long as they bring in enough high SAT students. The numbers do suggest that we could lose a lot of minority teacher candidates is admission committees aren’t paying attention to the situation.
(Thanks go to Brian Jacob for nudging me to make this table.)
What happens when schools face outside pressure to improve? Do they make real changes or do they just game the system to look better? Ceci Rouse, Jane Hannaway, Dan Goldhaber, and David Figlio show that schools really do change their ways.
Rouse et. al. sent surveys to every Florida elementary school for a number of years which let them track the details of what went on inside the schools. During this period, Florida implemented its “letter grade” system for grading every school on an A-F scale. Schools flunking with an F were publicly embarrassed, given some help to improve, and faced the threat of vouchers to let parents send their kids elsewhere.
Did schools game the system? Yup. For example some schools concentrated resources on just those students likely to improve enough to help the school over the threshold to a better letter grade.
But the authors find that the F-schools also made real changes. Here are some of the changes that F-schools made
- Put in policies to help low-performing students
- Lengthened instructional time
- Changed scheduling systems
- Put in policies to help low-performing teachers
- Gave teachers more resources
Not all these changes were large. And the fact that schools made changes doesn’t necessarily mean the changes were successful. But it’s good to know that the responses were real.
Michael Hansen has taken a look at “turn around” schools in North Carolina and Florida, compared them to other “chronically low performing” schools in those states, and decomposed the improvement in the turn around schools into three factors:
- Better performance from the teachers who remain in the school through the turnaround.
- Moving in high performing teachers.
- Moving out low performing teachers.
Turns out (1) matters a lot; (2) matters a bit; and (3) matters very little.
There has, I think, been too much attention given to firing really crummy teachers. Yes, we should do that. But there aren’t that many really crummy teachers. Our energy ought to be saved for boosting the vast portion of the teacher corps that is not crummy at all. And Hansen’s results suggest that’s just what happened in schools that underwent successful turnarounds.
We’d get a lot more of the great teachers we need if we paid ‘em better. If I’d substituted any other career for “teacher,” this would be a completely uncontroversial statement. (Okay, maybe teachers and clergy are thought of similarly.) For those attached to the potential-teachers-don’t-care-about-money fantasy, I’ve come across an interesting piece of evidence.
The non-profit arm of uber-consulting firm McKinsey & Company conducted a market research survey in which they asked people in the top third of college graduates the following kind of question: If we raised starting salaries would you decide to become a teacher; if the school working environment were better would you become a teacher? The answers showed that compensation matters a lot.
So a really big increase in both starting salary and maximum salary would increase the fraction of top-third students going into teaching by more than half.
The first conclusion from this study is that potential teachers do say that money matters.
But I want to go a little further. If you look at the table it looks like the magnitude of the salary increase needed to draw enough of the top-third of graduates into the profession is too high to be achievable. But the little “note” McKinsey included at the bottom is a reminder that the questions asked about salary changes holding constant other changes. If teaching salaries were higher, then the prestige of teaching would be higher…and the working conditions would be better…etc. So there’s a “multiplier effect” of higher salaries.
So the McKinsey study is good evidence, not conclusive but good evidence, that higher salaries would go a long way toward attracting a lot more of the best and the brightest into teaching.
Edvance Research reports on exciting results about the effect of Teach for America in Texas middle schools. The study’s authors used statistical techniques to put together groups of students with similar backgrounds. They looked at test score outcomes and compared TFA teachers to other inexperienced teachers They also compared TFA alums to other (non-TFA) experienced teachers. In several of the comparisons the TFA-taught students did enormously better.
- Middle school math: TFA beat other new teachers by a half year worth of learning.
- Middle school math: TFA alums beat other experienced teachers by nearly a full year of learning.
- Middle school reading: TFA alums beat other experienced teachers by a half year worth of learning.
In elementary school comparisons for math and reading both TFA-ers and alums beat their comparison groups, as did the middle school TFA-ers in math, but the results weren’t statistically significant. In other words the evidence for these groups also favors TFA, but the evidence isn’t strong enough for us to be very sure.
Mathew Chingos and Paul Peterson have a nice piece “The Impact of School Vouchers on College Enrollment” in Education Next where they look at the effects (in terms of later college attendance) of providing modest size vouchers to poor kids in New York City. The two cent summary (you really should go read the original) is that African-American kids got a big boost from the vouchers but nothing much happened for Hispanic students.
Chingos and Peterson’s findings (which are similar to what other researchers have found) made me wonder how this fits into the weird argument over whether good schools can overcome poverty–or whether everything is hopeless unless we can eliminate child-poverty.
Here’s an experiment which for at least one group showed that getting kids to more effective schools did make a big difference: even though nothing changed about the kids’ home background. So score one for the schools-do-matter crowd.
But there is one important caveat. The experiment compared results for kids who won a lottery to attend a better school with kids who lost out. Both sets of kids had parents with the get-up-and-go to at least apply for the vouchers. So this particular experiment doesn’t necessarily tell us what better schools do for kids from homes with less education-active parents.