a chalkbeat cheat sheet

Do school vouchers ‘work’? As the debate heats up, here’s what research really says

PHOTO: U.S. Department of Education
U.S. Education Secretary Betsy DeVos visiting the Christian Academy for Reaching Excellence in Miami.

The heated national debate about whether families should get public money to send their kids to private schools is full of big questions.

Do vouchers raise test scores or lower them? Do they help or hurt students over the long term? Do they damage public schools or push them to improve?

Chalkbeat combed through some of the most rigorous academic studies to get the answers.

Two caveats before we begin: First, context matters. Researchers look at specific programs at certain times with their own sets of rules. New initiatives — like a dramatic expansion of private school choice programs of the sort the Trump administration has promised — could mean entering uncharted waters, where past research becomes a less reliable guide.

Second, the voucher debate is often based on values. Research studies can’t answer philosophical questions on whether public money should go to religious schools or if providing more choices for parents is an inherent good.

With that in mind, here’s what you should know:

Recent studies suggest that vouchers lower student test scores in the short-run. But students who stay in private schools sometimes improve over time.

In the last couple of years, a spate of studies have shown that voucher programs in Indiana, Louisiana, Ohio, and Washington D.C. hurt student achievement — often causing moderate to large declines.

In Louisiana, after two years in the program, a student who started at 53rd percentile of performance had dropped to the 37th percentile in math. In D.C., students using a voucher fell seven percentile points in math and five points in reading after one year, compared to students who applied for a voucher but didn’t get one.

Advocates have pushed back, saying the programs were new and should be given more time to prove they work. Studies out of Indiana and Louisiana give some credence to that view.

In Indiana, students in the program saw initial dips in math achievement, but by year four those still in private school had caught up to their public-school peers. And in English, voucher students actually seemed to make gains after four years. These results, though, only applied to those who stayed in the program for four consecutive years.

In Louisiana, students in the upper elementary school grades saw huge test score drops in years one and two. Students caught up by year three in math and reading, though they still appeared to lag behind in social studies. Younger elementary students saw consistently big test score drops in math and social studies, though for methodological reasons, the researchers were less confident in these results.

The Ohio study showed that even three years into the program, the negative impacts of using a private school voucher persisted.

Older studies tended to show neutral or modest positive effects of vouchers on academic performance, and until recently, few if any studies had shown that vouchers actually led to lower achievement among students who received them. It’s not clear what has changed, but one theory is that public schools have improved — or at least gotten better at raising test scores — in response to accountability measures like No Child Left Behind.

PHOTO: Dylan Peers McCoy
Students sewing during a class at the School for Community Learning, a progressive Indianapolis private school that depends on vouchers.

A handful of studies show that vouchers have a positive or neutral impact on student outcomes later in life, like attending college or graduating high school.

Some supporters of vouchers downplay the recent studies by pointing to research on the longer-run effects of the programs. Here the studies are more positive, but they’re also limited and generally older. In each case, the researchers looked at students who entered voucher programs many years ago — a necessary trade-off when examining long-term outcomes.

A study of Milwaukee’s long-running voucher program found that participants were more likely to graduate high school and attend four-year colleges. A 2010 federal analysis of the D.C. voucher program found that its students were 21 percentage points more likely to complete high school (according to a survey of their parents, not a direct measure of graduation). A private school scholarship program in New York did not lead to improvements in college enrollment on average, but did seem to have a positive effect for black and Hispanic students specifically. A 2017 study of Florida’s school tax credit, the largest private school choice program in the country, found that it led to increases in college enrollment but had limited effects on degree completion.

It remains an open question whether the more recent initiatives can expect those results. The four programs weren’t hurting test scores in the short term; all had positive or no effects on scores.

Vouchers or tax credit programs lead to test score gains in public schools.

There is a large body of evidence suggesting that public schools improve in response to competition from school voucher or tax credit programs, at least as measured by test scores.

This has been seen in studies of Florida, Louisiana, Milwaukee, Ohio, San Antonio, and even Canada. As a recent research overview put it, “Evidence on both small-scale and large-scale programs suggests that competition induced by vouchers leads public schools to improve.”

The impact, though, is often fairly small and can dissipate over time. There do not appear to be any studies on the effect of voucher competition on measures other than test scores.

PHOTO: U.S. Department of Education

Attending a private school may improve parent satisfaction, particularly when it comes to school safety.

Families tend to be more satisfied with private schools, though it’s not clear why.

The older D.C. study showed that families who received vouchers had higher rates of parent (but not student) satisfaction. The latest D.C. analysis showed parents perceived private schools as safer and seemed more satisfied with them overall, though the results weren’t statistically significant.

An older report by the Manhattan Institute, a conservative think tank, found that families of students with disabilities using a voucher in Florida were dramatically more satisfied with the new private school than their previous public school. A recent analysis of national data showed that private-school parents were also more satisfied than those sending their children to public schools — though it could not establish cause and effect.

We know very little about how tax credit tuition programs affect students who participate.

Despite their expansion in recent years, tax credit programs — which use generous tax breaks to incentivize donations to organizations that then offer private school scholarships — have rarely been studied.

These programs are similar to school vouchers, in that they redirect taxpayer dollars to private schools, but they tend to be significantly less regulated. For instance, they usually do not require participating private schools to take state tests or, in some cases, any standardized tests at all. That’s part of why there’s so little research on how using one of those scholarships affects students.

The only program that has been rigorously studied, Florida’s tax credit scholarship, had either no effect or small positive effects on student test scores. But that was only prior to 2010; since then, changes to testing requirements have made direct comparisons to public school students impossible. A more recent study looking at students who entered the program between 2004 and 2010 found that it boosted college attendance.

Voucher programs targeted at low-income students are not likely to increase segregation — but a larger-scale program might.

There is surprisingly little research on the effects of private school choice programs on segregation. Existing studies have not found evidence that voucher programs targeted at low-income students worsen segregation. A recent study in Louisiana pointed to positive effects of vouchers on integration, while an older analysis of Milwaukee showed no impact.

However — based on research in other countries and other school choice initiatives including charter schools — there is reason to worry that a large-scale voucher program open to families regardless of income would exacerbate segregation. Prominent school choice advocates generally back a dramatic expansion of this sort.

choice challenge

A Betsy DeVos-approved tax change is meant to make private school more affordable. Here’s why it might not work

PHOTO: Department of Education
U.S. Education Secretary Betsy DeVos.

The just-passed tax law includes a big perk for families who send their children to private school: the ability to use certain tax-advantaged savings accounts, which until now could only be used to save for higher education, to pay for K-12 school, too.

That’s led to criticism from those who note that the wealthy stand to benefit the most and that local public school budgets could take a hit, and support from school choice advocates like Betsy DeVos, who argue it will expand access to private schools.

But there’s reason to believe that the move won’t make private school feasible for any more families — and that private schools are likely to raise tuition in response.

Here’s why: All families are eligible to use the saving accounts, known as 529s. That means unlike a lot of state programs that offer private school vouchers or tax credits, the 529s aren’t targeted at poor students, those with a disability, or other specific groups.

In a peer-reviewed 2016 study, researchers compared the effects of these two types of choice programs: those that are restricted to certain populations and those available to all. The latter group — that is, programs like 529s — didn’t lead to any increase in students attending private school, but did cause a sharp hike in school tuition.

This suggests these programs don’t function the way some advocates want them to. Instead of adding choices for families, they offer a windfall to schools. (Keep in mind that students could still benefit if private schools use that extra money to improve the quality of education they offer.)

There is one important reason why private schools may not react to this change in the same way. Although all families will technically be able to use the new 529 rules to save for private school, in practice, only families with enough money to put extra into one of the accounts will be able to participate. In that sense, it could function more like a targeted program, and those have been found to boost private school attendance.

In praising the initiative on Tuesday, DeVos acknowledged the fact that the program is unlikely to benefit poor families.

“Anything that empowers parents and gives them more opportunities for their students is a good thing,” she said. “But it doesn’t address the needs of parents who are from lower incomes and doesn’t empower them in significant ways.”

devos watch

Four takeaways from Betsy DeVos’s summit on innovation in K-12 education

Education Secretary Betsy DeVos earlier this month. (U.S. Department of Education)

Betsy DeVos used her bully pulpit on Tuesday to again call for more school innovation, especially technology-infused “personalized learning.”

“Washington, D.C. does not have all the answers,” DeVos said at the Department of Education’s K-12 focused “Rethink Schools Summit.” “But government can be good at bringing people together to highlight their creative thinking and new approaches.”

It’s a familiar tactic for DeVos, who has been largely stymied in pushing school choice policies but has repeatedly put the spotlight on specific schools she finds innovative. Charter and private schools were well represented at the Tuesday meeting, which also included a number of district leaders and an array of others, including arts-education, homeschooling, and community-schools advocates.

Here are a few takeaways from the event:

Generalities outnumbered policy specifics.

There appeared to be broad agreement in the room on several general notions: Teachers are important, technology needs to be used, but used wisely, and schools must change to prepare students for a changing world.

“Technology is the not the answer,” said Tom Rooney, the superintendent of Lindsay Unified School District in California that has embraced a tech-infused approach that he calls customized learning. It’s “about using technology in transformational way to accelerate the learning.”

Rooney pointed to an effort to expand wireless Internet throughout the community, and to schools that group students by performance levels, rather than age or grade.

But DeVos’s request for participants describe “where impediments at any level of government are preventing you from achieving your mission” went largely unheeded. Two exceptions were concerns raised about state testing requirements and teacher certification rules.

Personalized learning was front and center, dovetailing with the goals of some big philanthropists.

The theme of the day was the notion of tailoring teaching to individual students — alternatively called personalized, customized, or student-centered learning.

This approach aligns with the agendas of several influential education foundations, namely the Emerson Collective, the Bill and Melinda Gates Foundation, and the Chan-Zuckerberg Initiative. (Emerson and Gates are both funders of Chalkbeat.)

Among the speakers was Diane Tavenner, the founder of Summit, a charter school network that also provides software to district schools to support tech-based personalized learning. Summit has been supported by the Gates Foundation, won an XQ prize from the Emerson Collective, and is backed by Chan-Zuckerberg.

Mark Zuckerberg specifically mentioned the group in a recent post describing his approach to charitable giving. “Our partnership with Summit Public Schools has helped encode their teaching philosophy in tools that will be used in more than 300 district, charter, and private schools this fall,” he wrote.

The leader of the Grand Rapids Public Museum High School, which also won a $10 million XQ prize, was present, as was a representative of Leap Innovations, a nonprofit group that consults on personalized learning and has been praised by Bill Gates and Jim Shelton of Chan-Zuckerberg.

Some notable players weren’t present: prominent charters and virtual schools.

Those not at the summit included fully virtual charter school operators — like K12 or Connections Academy — who DeVos and some other school choice advocates have praised as innovative, but that research has found lead to large drops in student achievement.

Also missing: high-profile “no-excuses” charter networks, such as Achievement First, KIPP, or Success Academy, which have posted consistently high test scores.

There were bold claims of success, but little new evidence.

DeVos and other participants appeared confident that their focus on personalized learning will succeed, despite the limited evidence to date. Participants made big assertions, including district leaders saying that test scores and graduation rates had improved after moving to a different approach. But it’s difficult to say whether a more personalized approach deserves credit for these gains, and there was little reference to research evidence throughout the conference.

Summit, for instance, has not been the subject of much rigorous external study. The closest may be a recent report by Stanford’s CREDO, which only examined about 400 students in the Summit network. It found the schools had no statistically significant effect on reading test scores and small negative impacts in math. Summit has produced an internal analysis showing that students using its software made faster than average growth on a national test.

In general, personalized learning advocates highlight older research on the benefits of one-on-one tutoring, studies finding that specific math-focused computer programs can lead to gains, and a RAND report pointing to gains in schools that have adopted personalized learning. The RAND researchers, though, have urged caution.

“I worry that the positive results that have come out of our studies are generating a bit too much enthusiasm,” RAND researcher John Pane told Education Week in November. “I think people see the headlines, but they don’t see the limitations of the research that’s happened so far.”

Here’s the full list of participants:

  • Mashea Ashton of Digital Pioneers Academy
  • Nicole Assisi of Thrive Charter Schools
  • Carol Becker of Homeschool Cooperative
  • Beth Blaufuss of Archbishop Carroll High School
  • Michael Bolling of CodeRVA
  • Patricia Brantley of Friendship Public Charter Schools
  • Jean-Paul Cadet of Prince George’s County Public Schools
  • Heather Clawson of Communities in Schools
  • Elizabeth Goettl of Cristo Rey Network
  • Kamal Hamdan of California State University and STEM Lab School
  • Christopher Hanks of Grand Rapids Public Museum School
  • Andrew Hart of The Oaks Academy
  • Chris Liang-Vergara of Leap Innovations
  • Stephen Mauney of Mooresville Graded School District
  • Carol Morgan of ArtsConnection
  • Tom Rooney of Lindsay Unified School District
  • Vielka Scott-Marcus of Friendship Public Charter Schools
  • John Swoyer III of MaST Community Charter School
  • Diane Tavenner of Summit Learning
  • Ken Wagner of Rhode Island Department of Education
  • Travis Works of Cornville Regional Charter School
  • Doug Wright of Carroll County Schools