Good news: a failing charter school is shut down

The key idea behind charter schools is that good ones should be expanded and bad ones shut down. This is working in East Palo Alto, where the school board has voted to shut down a charter that is the worst-performing school in the district. The school is run by the Stanford Ed School and was founded by Linda Darling-Hammond, a prominent teacher educator. One of the highest-performing schools in the district is also a charter school, run by Aspire. Ironically, the Stanford team partnered with Aspire when it initially launched its charter. The partnership collapsed, reportedly because Aspire was too focused on academics for Stanford. The schools went their own ways, with vastly different results. http://www.nytimes.com/2010/04/16/education/16sfcharter.html

Weil & the "Evidence of Experience."

Reading Denialism: How Irrational Thinking Hinders Scientific Progress, Harms the Planet, and Threatens Our Lives, by Michael Specter. Fun and enlightening read, I recommend it. I don't agree with everything he says, but it got me thinking, and I certainly value that in a book. His chapter on alternative medicine moved me from thinking to blogging. Specter quotes health guru Andrew Weil on randomized trials:

There is a great movement toward "evidence-based medicine" today, an attempt to weed out ideas and practices not supported by the kind of evidence that doctors like best: results of randomized trials. This way of thinking discounts the evidence of experience.

Weil misses the point here quite quite badly. A randomized trial aggregates the experiences of the dozens or hundreds or thousands of people who participated in the experiment. By valuing the findings of a randomized trial more than the intuition generated by our own narrow worldview, we are paying attention to the experiences of others. Counterquote from Dara O'Briain, Irish standup comic:

Just because science doesn't know everything doesn't mean you can fill up the gaps with whatever fairy tale most appeals to you.

You can see Dara's routine here.

Charter Schools Close the Achievement Gap

I have been happily working with a killer set of co-authors recently: Josh Angrist, Parag Pathak and Chris Walters of MIT, Tom Kane of Harvard and Atila Abdulkadiroglu of Duke. We are writing a series of papers that evaluate the effect of charter schools on test scores. We are focused on Massachusetts, where we have an excellent working relationship with its very capable office of education research, which has allowed us access to the statewide test score data needed to undertake an evaluation of this nature.

The key empirical challenge in understanding the effect of charter schools is selection bias: kids who go to charter schools are different in both observable and unobservable ways from kids who don't. You can try to control for the visible differences, as does this study from Stanford. But what we can measure in terms of student background is pretty scant in administrative datasets: sex, race, free/reduced lunch eligibility, special-education status, previous test scores. So there remains plenty of risk for omitted-variable bias. Are kids whose parents are highly educated or motivated concentrated at charters? Kids whose test scores were plummeting in the public schools? Kids who were not challenged in the public schools? All of these differences would contaminate any effort to compare the achievement of kids at charters and kids at public schools.

We solve this problem by exploiting the randomized lotteries conducted by over-subscribed charter schools. The lottery approach focuses on students who apply to charters, comparing outcomes for those who lose the lottery to those who win. A mere coin flip (or randomly-generated number) separates the lottery winners and losers, so we can be confident that they are alike in every observable and unobservable way - except for their charter school attendance. This closely approximates the gold standard of a randomized, controlled trial.

We used this approach to study charters schools in Boston, and have just written a paper focused on a KIPP charter in Lynn, a working-class city north of Boston.* A policy report and two academic papers (on Boston and KIPP Lynn) have been born so far of this collaboration. All our results point to large and positive effects of charter schools on student achievement. The effects are larger for kids who most need help: Blacks, Hispanics, those with limited English proficiency, special ed kids, those who have the lowest baseline scores. The effect sizes are huge - kids at charters gain 0.1-0.2 standard deviations each year on their peers at the traditional public schools. How "big" is this? Wicked big, as they say in my hometown of Somerville, a few miles south of Lynn. In Lynn, the Hispanic-White test score gap is 0.5 standard deviations, as is the Black-White test score gap. Interventions that come anywhere close to gains of that size are few and far between. By way of comparison, the STAR experiment found that smaller classes (sustained for four years, from kindergarten through third grade) increased test scores about 0.2 standard deviations.

We are not the first to use the lottery approach; Caroline Hoxby, Jonah Rockoff and co-authors have looked at charter schools in Chicago and New York City using this methodology, and Roland Fryer has used this approach to examine charters in the Harlem Children's Zone. But the KIPP Lynn paper provides the first lottery evidence on a charter school that serves a large Hispanic population (the school is about half Hispanic): the New York City, Chicago and Boston charters are predominantly Black, and enroll fewer Hispanics that the surrounding schools. And it is the first randomized evaluation of a KIPP charter, which are spreading across the nation. The KIPP results are convincing evidence that charters can succeed with students who have the greatest need. A recent UFT report raised the concern that charters are failing (or ignoring) these very populations. At least in Lynn, KIPP is achieving astounding results with these kids.

..................................................................................

*That's the Lynn in the chant sung to generations of children being bounced on their parents' laps: "Ride a horse to Boston, ride a horse to Lynn, better watch out or you're gonna fall IN!"