jonathan lovell's blog

a site to explore the difference between meaningful and ill-conceived educational reform (note: you have to open a post for its links to be activated)

“improving educational outcomes is a hard, messy, complicated process”

This thoughtful article on Sweden’s ten years’ experiment with “choice” in public education, by Ray Fisman, Professor of Economics at the Columbia Business School, places our own nation’s infatuation with “choice” in a useful context. As Fisman writes at the end of this article, “Simply opening the floodgates to more education entrepreneurs doesn’t disrupt education. It’s just plain disruptive.”

Enjoy the read!


Sweden’s School Choice Disaster

Every three years, Americans wring their hands over the state of our schools compared with those in other countries. The occasion is the triennial release of global scholastic achievement rankings based on exams administered by the Program for International Student Assessment, or PISA, which tests students in 65 countries in math, science, and languages. Across all subjects, America ranked squarely in the middle of the pack when the tests were first given in 2000, and its position hardly budged over the next dozen years.

The angst over U.S. student performance—and its implications for the American workforce of the near future—is inevitably accompanied by calls for education reform: greater accountability, more innovation. Just as inevitable are the suggestions for how more accountability and innovation could be realized: more charter schools, more choice, less bureaucratic oversight.

Advocates for choice-based solutions should take a look at what’s happened to schools in Sweden, where parents and educators would be thrilled to trade their country’s steep drop in PISA scores over the past 10 years for America’s middling but consistent results. What’s caused the recent crisis in Swedish education? Researchers and policy analysts are increasingly pointing the finger at many of the choice-oriented reforms that are being championed as the way forward for American schools. While this doesn’t necessarily mean that adding more accountability and discipline to American schools would be a bad thing, it does hint at the many headaches that can come from trying to do so by aggressively introducing marketlike competition to education.

To the extent that Americans think about Sweden at all, it usually conjures images of gibberish-speaking Muppets, Ikea furniture, and, when it comes to government policies, the welfare state. So it’s something of a surprise that a higher fraction of Swedish students go to privately run (and mostly for-profit) schools than in the U.S.* The system was put in place in the early 1990s by a center-right (by Swedish standards) government, inspired by the ideas of the godfather of free market economics, Milton Friedman. In a 1955 article titled “The Role of the Government in Education,” Friedman advocated for a system in which governments would issue vouchers to parents that would be redeemable toward tuition payments at a private school of their choice. This voucher system would allow market pressures to work their magic, as schools would be forced to improve their quality in order to compete for students and their voucher dollars. (Even if governments remained in the education business, public schools would face the same market pressures to maintain enrollment as private schools.)

There are differences between the libertarian ideal espoused by Friedman and the actual voucher program the Swedes put in place in the early ’90s. Friedman would have allowed schools to charge parents more in tuition than what a voucher could cover, potentially allowing rich parents to send their kids to better-resourced schools than poor parents could. Friedman’s system also allowed for choice on both sides of the market: Just as parents got to pick where to send their kids, schools had the right to accept some applicants and reject others. In the name of equal access, Sweden forbade nonvoucher tuition payments and permitted only limited screening (based on scholastic performance) for high school admission.

But Swedish school reforms did incorporate the essential features of the voucher system advocated by Friedman. The hope was that schools would have clear financial incentives to provide a better education and could be more responsive to customer (i.e., parental) needs and wants when freed from the burden imposed by a centralized bureaucracy. And the Swedish market for education was open to all, meaning any entrepreneur, whether motivated by religious beliefs, social concern, or the almighty dollar, could launch a school as long as he could maintain its accreditation and attract “paying” customers.

For a while, at least if media accounts of the reforms are any indication, things looked like they were going pretty well. Voucher school students consistently outperformed their counterparts at government schools; in 2008, the London Telegraph described the reforms’ impact as “tremendous.” The number of private schools increased tenfold in less than a decade, with a majority run as for-profits.

But in the wake of the country’s nose dive in the PISA rankings, there’s widespread recognition that something’s wrong with Swedish schooling. As part of ongoing efforts to determine the root cause, the Swedish Schools Inspectorate (the equivalent of the U.S. federal government’s Department of Education) called for a regrading of a subset of standardized tests administered during 2010 and 2011. In total, nearly 50,000 students at all grade levels from more than 700 schools had their tests in English, Swedish, science, and math re-evaluated.

Two Stockholm University economists, Björn Tyrefors Hinnerich and Jonas Vlachos, have been analyzing the data, and their findings to date demonstrate the many ways that things can go wrong under a market-driven education system. As in many countries, Sweden has standardized tests that are administered to all students nationwide. Performance matters for both students and the schools they attend. Students who do well will have brighter admission prospects. Schools that do well attract more (and perhaps better) students; the ones that perform poorly risk losing accreditation. However, unlike, say, the SAT, which is sent off to be graded at the testing service’s headquarters, in Sweden grading is done locally, often by teachers, often at the school where the test-takers are enrolled. (This setup is not uncommon in the U.S. as well: The New York state Regents Exam is administered to all students and graded by each student’s own teacher.)

It’s easy to imagine teachers going easy on their own students, possibly unconsciously, motivated by nothing more than a desire to help them—a study co-authored by my colleague, Jonah Rockoff, found exactly this result in an analysis of New York Regents Exam grading. In Sweden, according to the study by Hinnerich and Vlachos, the scores issued by external evaluators were indeed harsher than those assigned by internal graders. And after accounting for things like a school’s location, along with basic student characteristics, it turned out that the external evaluators had downgraded the scores for students at voucher schools much more than for students at government ones. In fact, a sizable portion of the much-vaunted outperformance of voucher school students could be chalked up to nothing more than easy grading. More surprising still, the voucher school grade inflation is almost as high for math and science (where you’d think an answer is either right or wrong) as it is for Swedish. By contrast, Rockoff et al.’s Regents Exam analysis found very little evidence of test score manipulation in quantitative subjects.

It’s the darker side of competition that Milton Friedman and his free-market disciples tend to downplay: If parents value high test scores, you can compete for voucher dollars by hiring better teachers and providing a better education—or by going easy in grading national tests. Competition was also meant to discipline government schools by forcing them to up their game to maintain their enrollments, but it may have instead led to a race to the bottom as they too started grading generously to keep their students.

None of this is terribly surprising—in econ 101 we learn that markets work their magic when buyers and sellers are well-informed about what’s getting bought and sold, and can therefore transact with one another without fear of getting conned. The apparent failure of the Swedish schooling experiment is a lesson in the inability of markets to solve problems where it’s hard to compare the educational “product” that’s offered, and the outcomes you can observe are subject to manipulation. It’s also a reminder that the cold, hard calculations of markets aren’t necessarily suited to the realm of education. Governments don’t shut schools because they fail to turn a profit. Private equity firms do. The parents of more than 10,000 students learned this difference the hard way last year, when the Danish private equity group Axcel abruptly announced its exit from the Swedish school market, stating that it could no longer cover the continued losses.

What can Americans learn about education reform from the Swedish experiment? It would be wrong to blame all of Sweden’s education problems on voucher schools—even with their rapid expansion, only about a quarter of secondary-school students are enrolled at private schools; among elementary schools, the rate is only about 13 percent. Yet the way they seem to have abetted teaching to the test and otherwise thrown Swedish education off course suggests they may be more of a problem than part of a solution.

Even in a place like New York City, where charter schools have proven to be popular and successful, they enroll less than 5 percent of the city’s 1 million students. The city maintains control over its numbers through the issuing of a fixed number of charters. It’s easier for New York’s Department of Education to watch more carefully over hundreds (rather than thousands) of charter schools, pressuring the ones that are underperforming and shutting them down if they can’t turn things around. Sweden’s problems should temper the enthusiasm of reformers for free entry of new schools or even large-scale charter expansion. When markets are prone to failure, it’s probably best for regulators to maintain some oversight and control.

None of this should be taken as a critique of the many high-performing charter schools in this country—the KIPP academies, the Uncommon Schools, and so forth—where solid research has shown that students who get lotteried into one of their classrooms are vastly more likely to finish high school and enroll in college, relative to kids who applied but weren’t lucky enough to get in. There’s surely a lot that can be learned from trying to uncover the secret formula that makes these high-performing charters do so well, ideas that can then be applied more broadly in public and charter schools alike.

This “innovation engine” view of charter schools is the more modest position advocated by people like Norman Atkins, president of the Relay Graduate School of Education (and the founder of Uncommon Schools). “Charter schools are the Silicon Valley of Education,” Atkins told me via email, with charters incubating practices that can later spread across all schools.

For students in some American cities, a school system dominated by charter-based school choice isn’t a hypothetical—it’s already arrived. Most notably, more than 90 percent of New Orleans students now attend a charter school, the result of a transformation that began more than a decade ago and accelerated in the post-Katrina years. While the state-run Recovery School District created in 2003 has its share of critics, credible external evaluation suggest that test scores and graduation rates have indeed improved under the districtwide takeover by charter schools. How are these improvements squared with the Swedish experience?

I posed the question to Maggie Runyan-Shefa, who serves as co-CEO of New Schools for New Orleans, the organization leading reform efforts in the Recovery School District. She began by observing that the school system was in such dire straits a decade or so ago that, “something radical was needed.” Unlike Sweden, which was taking a perfectly well-functioning school system and handing it over to private operators, New Orleans was starting from rock bottom. (To drive this point home, Runyan-Shefa’s predecessor, Neerav Kingsland, liked to tell the story of a class valedictorian who failed to graduate after flunking a math competency test five times.) There was a willingness to take drastic action, and, frankly, the schools had nowhere to go but up.

In Runyan-Shefa’s telling, the functioning of New Orleans’ schools also departs in a critical way from a pure voucher system. As is the case in New York City, the government maintains an active regulatory role, something that surely would have Milton Friedman turning in his grave. District administrators, rather than market pressures, decide which schools stay open and which ones are forced to close. While the same was true in theory in Sweden, in practice the government rarely revoked voucher schools’ accreditations.

At the same time, even if we accept New Orleans as a success story, it’s fair to ask whether similar success might have been achieved through a thorough reform of a traditional public school system. The much-vaunted schools of Finland, a country entirely free of charter schools, consistently perform near the top of the PISA rankings. There are surely many roads to scholastic salvation. And Runyan-Shefa agreed that there would be many challenges in scaling up the New Orleans model nationally. For example, one of their reform strategies involved attracting talent from across the country, which by definition can’t work once everyone’s doing it. (Recognizing that the pool of quality educators nationwide is limited, Runyan-Shefa said they are also working to develop homegrown talent.)

Maybe the overall message is, as Norman Atkins of Relay GSE put it to me, “there are no panaceas” in public education. We tend to look for the silver bullet—whether it’s the glories of the market or the techno-utopian aspirations of education technology—when in fact improving educational outcomes is a hard, messy, complicated process. It’s a lesson that Swedish parents and students have learned all too well: Simply opening the floodgates to more education entrepreneurs doesn’t disrupt education. It’s just plain disruptive.

*Correction, July 22, 2014: The article originally stated that more Swedish students go to privately run (and mostly for-profit) schools than in any other developed country on earth. In fact, other countries, like South Korea and the Netherlands, send a higher fraction of students to privately run schools.

Ray Fisman is a professor of economics at the Columbia Business School and co-author of The Org: The Underlying Logic of the Office. This article was initially posted on Slate July 15 2014 at 11:48 PM

Categories: Uncategorized

1 reply


  1. “improving educational outcomes is a hard, messy, complicated process” | jonathan's edutalk updated

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s