I have a new paper with my Cambridge collaborators on the effects of Boston's charter schools on preparation for college and college choice. In previous work (which I blogged about here), we looked at the effects of these schools on the state's standardized test, the MCAS. We found large and positive effects of charter attendance, with effects largest for kids who most need help: Blacks, Hispanics, those with limited English proficiency, special ed kids, those who have the lowest baseline scores. The effect sizes are huge - kids at charters gain 0.1-0.2 standard deviations each year on their peers at the traditional public schools. This earlier paper on test scores has now been published in the Quarterly Journal of Economics.
Test scores are not what makes the world go round, however - the aim of education is to make better, smarter, happier, well-rounded citizens. While we don't measure all of these things in our new work, we gain some ground by examining preparation for college (in the form of the SAT and Advanced Placement scores), college entry and choice of college. As the children attending these schools age, we hope to look at yet more outcomes.
Summary of the paper's findings:
We use admissions lotteries to estimate the effects of attendance at Boston's charter high schools on college preparation, college attendance, and college choice. Charter attendance increases pass rates on the high-stakes exam required for high school graduation in Massachusetts, with especially large effects on the likelihood of qualifying for a state-sponsored college scholarship. Charter attendance has little effect on the likelihood of taking the SAT, but shifts the distribution of scores rightward, moving students into higher quartiles of the state SAT score distribution. Boston's charter high schools also increase the likelihood of taking an Advanced Placement (AP) exam, the number of AP exams taken, and scores on AP Calculus tests.
Finally, charter attendance induces a substantial shift from two- to four-year institutions, though the effect on overall college enrollment is modest. The increase in four-year enrollment is concentrated among four-year public institutions in Massachusetts. The large gains generated by Boston's charter high schools are unlikely to be generated by changes in peer composition or other peer effects.
The numbers behind this summary are pretty impressive. Charter attendance increases SAT scores by about a third of a standard deviation and doubles the likelihood of taking and passing an Advanced Placement test. Charter attendance quadruples the likelihood of taking the AP calculus exam (from 6% to 27%) and quintuples the likelihood of getting a passing score (from 1.5% to 9%). As these numbers make clear, a lot of the kids induced to take an AP course don't end up getting college credit for it. The fact that they are able to take the class at all, however, indicates that they have taken a strong set of college-prep classes. In particular, you can't take calculus without having taken algebra, trigonometry and geometry - all of which you need to be in the running for a selective, four-year college and a STEM career.
Charter attendance also affects postsecondary outcomes. Most strikingly, kids who attend Boston's charters are 17 percentage points more likely than their comparable peers to attend a four-year college. There also appears to be a positive effect on attending any college at all, but these estimates are not precise enough to take to the bank. We have to wait for more cohorts of these kids to age into college before we can say anything definitive on this point.
I discussed why we use admissions lotteries to get at these results in my earlier post. To recap: the key empirical challenge in understanding the effect of charter schools is selection bias: kids who go to charter schools are different in both observable and unobservable ways from kids who don't. Are kids whose parents are highly educated or motivated concentrated at charters? Kids whose test scores were plummeting in the public schools? Kids who were not challenged in the public schools? All of these differences would contaminate any effort to compare the achievement of kids at charters and kids at public schools.
We solve this problem by exploiting the randomized lotteries conducted by over-subscribed charter schools. The lottery approach focuses on students who apply to charters, comparing outcomes for those who lose the lottery to those who win. A mere coin flip (or randomly-generated number) separates the lottery winners and losers, so we can be confident that they are alike in every observable and unobservable way - except for their charter school attendance. This closely approximates the gold standard of a randomized, controlled trial. The What Works Clearinghouse has reviewed an earlier version of our paper and given it a provisional "Meets Standards without Reservation."
