voucher verdict

Students’ math scores drop for years after using a private school voucher in country’s largest program

Ft. Wayne, IN: Students walk past "The Road to Success" sign displayed in the hallway at Horizon Christian Academy in Ft. Wayne, Indiana December 20, 2016. Horizon Christian Academy is one of more than 300 schools that accepts vouchers in Indiana, which has the largest statewide voucher program in the nation.

Low-income students who use a voucher to attend private school in Indiana see their math scores drop for several years as a result, according to a new study.

The findings are a blow to the argument that poor students benefit from the choice to attend a private school, a policy championed by U.S. Secretary of Education Betsy DeVos.

“Our results do not provide robust support that the [voucher program] has been successful to date at improving student achievement for low-income students who use a voucher to switch from a public to a private school,” conclude the researchers, Mark Berends of Notre Dame and Joseph Waddington of the University of Kentucky.

The paper, focuses on the initial rollout of what has become the largest school voucher program in the country. In the most recent school year, over 35,000 students were enrolled in the initiative.

The study examines a few thousand low-income students who switched from public to private school using a voucher starting in the 2011-12 school year.

Notably, the authors show that low-income students who used a voucher had slightly higher starting test scores than low-income kids who stayed in public schools. This gives credence to fears that a voucher program could concentrate the most disadvantaged students in the public school system.

The authors attempt to control for these and other factors to isolate the effect of attending a private school. (Unlike some voucher studies, this paper is not able to compare students who randomly won or lost a chance to attend private school — a stronger method.)

In math, the results, which focus on grades five through eight, are consistently negative. Even four years into the program, students who use a voucher had lower test scores than public school students.

In English, there were no clear effects. Here, there was some evidence that voucher students improved over time, though there were no statistically significant positive effects after four years.

The results were generally consistent for students of different races, genders, and locations. The findings, though, might not hold for more affluent students or the increasing numbers of participating students who never attended a public school to begin with. Neither group is accounted for in this study.

The results, published this week in the peer-reviewed Journal of Policy Analysis and Management, are largely in line with an earlier version of the same study — with a key exception. The first paper suggested that declines in math disappeared for students who used a voucher for multiple years. The latest version finds that the negative effects seem to persist for at least four years.

The results, then, undermine the argument that recent studies showing drops in voucher students’ test scores just represent students adjusting to private school. A recent study in Washington, D.C. showed substantial drops in math achievement persisted into a second year. Negative results also were consistent in an Ohio voucher study. In Louisiana, math and reading test scores bounced back according to one method but not another after three years; drops were consistent in social studies.

Voucher advocates have responded by arguing that tests should not be the sole judge of these programs’ success, pointing to more favorable research looking at high school graduation and college enrollment. Despite test score declines, a recent study showed that Louisiana’s program had no effect or somewhat positive effects on college enrollment.

“Although academic achievement outcomes are important for researchers, policymakers, and practitioners to consider, parents make schooling decisions for their children based on a multitude of factors, including academics, location, safety, and religion,” Waddington and Berends write. “Therefore, researchers need to examine outcomes beyond test scores.”

Critics might also point to other concerns not captured in the study: A Chalkbeat investigation in 2017 found that about one in 10 Indiana private schools that accepted a voucher had policies that explicitly discriminated against LGBT students.

That’s perfectly legal under Indiana’s system, as well as the vast majority of publicly funded private school choice programs.

outside the box

How one Chicago principal is leaning on data to help black boys

PHOTO: Adeshina Emmanuel

Test scores were rising at Fuller Elementary School when Marilyn McCottrell took over in 2016. Yet troubling trends loomed behind the numbers.

“A lot of growth has been made,” said McCottrell, Fuller’s third principal in six years. “But that growth is not equal among students.”

She’s talking about black boys.

Black girls had driven most of Fuller’s academic improvement since the 2012-13 school year, when Chicago Public Schools handed management of the Bronzeville school over to the non-profit Academy for Urban School Leadership, which replaced the staff and principal in a turnaround effort. Black boys had improved much slower. They got most of the school’s Ds and Fs, and were much less likely than girls to meet or approach expectations for college readiness on state tests.

PARCC Scores

Last school year, McCottrell and her staff crunched the data and made changes at Fuller to shorten the gaps between boys and girls. The stakes are high. Black boys, especially those from low-income households, are more prone than their sisters to falling behind in school and running into the juvenile criminal justice system. As adults, they are more likely to be arrested, imprisoned, or chronically unemployed. McCottrell believes what Fuller did, starting with painstakingly crunching data at the school, classroom and individual levels, could help other schools do better for black boys.

But she wants to be clear about something: Black boys don’t need to be “saved.”

“They need to be respected and appreciated for the differences and the unique gifts that they bring to the educational experience,” she said.

Black boys
PHOTO: Adeshina Emmanuel
Fuller Elementary School students (from left)Tyrese Robinson-Guy, Terrell Johnson, and Jasean Waters at a community garden in Bronzeville.

 

Fuller, a Level 1 school in good standing, occupies the corner of St. Lawrence Avenue and 42nd Street in Bronzeville. Nearly all of its 370 students are black and come from economically disadvantaged households. When CPS turned over management of Fuller, it was seeking to lift up a school that had been on academic probation five consecutive years. Fuller still has far to go. In 2017, only 10 percent of Fuller students were ready for the next level compared to 26 percent across the school district and 34 percent across the state. Growth has been above average, but, as McCottrell said, that growth hasn’t been equal.

PARCC Scores

Last August, McCottrell arrived at Fuller for a training session for teachers bearing handouts packed with data on black boys’ grades and test scores. Middle school reading teacher Arlicia McClain was shocked to see the stark disparities.

“It made me buck up and say I need to talk to these students,” she said. “I need to know what is going on that is preventing them from improving. Is it me? Is it something going on with them individually? Is it something they are missing?”

Girls’ math scores had increased by 193 percent compared with 90 percent for boys since the turnaround effort began in the 2012-13 school year. The gender performance gap was even more striking in reading, where black girls’ scores jumped 140 percent compared with 31 percent for boys. 

As McClain and other teachers reflected on the numbers, they recounted their  own experiences in the classroom. For example, they could all name which students were removed from class the most for disciplinary reasons, and nearly all were black boys.

Arlicia McClain
PHOTO: Courtesy of Arlicia McClain
Fuller Elementary School teacher Arlicia McClain.

McClain realized she tended to call on black girls more in class.

McClain, African-American herself, wondered if she was favoring girls or failing to challenge boys enough, and how that could affect their learning. She resolved to push black boys more during her second year at Fuller.

She also left the session with another big take-away: A lot of boys who wouldn’t participate in classroom-wide sessions engaged more in small groups. Wedding the data to her realizations has helped the young teacher come up with tailored approaches for struggling students.

“Look at them as individuals who want to learn, but who sometimes need the individualized attention to do that,” McClain said. “If you really are about the progression of black youth, you’re going to need to be individual-focused, and you’re going to need the data to do it.”

In the 2016-17 school year, for the subjects of English language arts and math, about 70 percent of all Ds and Fs at Fuller went to black boys.

In the first quarter of last school year, McCottrell and her staff revised Fuller’s grading policies in hopes of addressing the disparity.

They switched to what McCottrell called “a more equitable grading scale,” where the lowest a student could score is a 50, adopted a “no-opt out policy” for homework, so children who failed to turn in their homework by deadline wouldn’t automatically get a zero and had to make up assignments, and allowed students to redo certain parts of failed tests and quizzes after reteaching.

By the end of the first quarter, the numbers of Ds and Fs had decreased by nearly half.

But black boys were still getting about the same percent of them as before.

So McCottrell decided to go in for a closer look.

“The numbers only tell part of the story,” she said.

McCottrell ate with boys in the lunchroom. She played flag football with them at recess. She sat with them in class, assisted their teachers, and taught her own lessons across grades and subjects.

She talked to the boys — and listened.

Jasean Waters, a black boy

Jasean Waters, 13, said he found it hard to focus on his school work.

Some distractions come from inside the classroom, like the bullies Jasean’s run into. Other distractions live in the world outside Fuller, like the gun violence whose victims are overwhelmingly black males.

“It’s a big struggle for us,” he said. “There’s a lot of people dying around here, so we gotta watch our backs, and when we’re walking home we feel like we’re unsafe, so we just focus on us being safe. It’s hard to focus on school.”

Boredom is another issue. Jasean said that he does well in math, but struggles sometimes with reading, and that his interest wanes with the lack of characters and authors he can relate to in school texts. That sounded familiar to McCottrell.  When she spoke with boys, she heard that school amounted to a seven-hour suppression of their personalities, interests, and voices — especially in reading and English classes, where black voices and black writers were missing.

“When kids have to pick a book for independent reading, they don’t relate to the characters in those classroom libraries,” she said. “It’s really hard coming to a class everyday when nothing relates to you.”

Parcc Scores

McCottrell decided to teach an optional African-American literature class every Friday during a weekly “intervention time” for students needing help in reading and math  About 17 boys showed up on the first day and read excerpts from Ralph Ellison’s “Invisible Man,” whose protagonist proclaims, “I am invisible, understand, simply because people refuse to see me.”

McCottrell said many of the boys could expertly analyze the Harlem Renaissance classic, because they related to the idea of not being heard, seen, or understood for who they really are. The students offered examples like the portrayal of black men in the media.

“Many of them were saying things like, ‘I’m not a gangbanger, but this is what people think I am, because I’m dark or because I’m tall,’” she said. “They talked about it in the context of their teachers not knowing who they are.”

The class soon doubled as word of mouth drew others in. Jasean, a C student at the start of the class, joined them. He said he learned things he hadn’t been introduced to before. He read about segregation, speeches by Martin Luther King, and books like “Bud, Not Buddy,” about a 10-year-old black orphan during the Great Depression.

He said he rededicated himself to doing 100 minutes of reading a night and by the end of last school year earned an A in reading. He said he raises his hand to ask and answer questions in class more.

“It feels good,” he said.

Jasean’s grandmother, local school council member Regina Waters, praised McCottrell’s hands-on approach with students and her efforts to build one-on-one relationships with the boys.

“She’s upfront with the kids, and she knows all the kids by name which is unusual in the short time she’s been there,” Waters said.

McCottrell
PHOTO: Adeshina Emmanuel
Marilyn McCottrell

Fuller’s boys closed the gap with girls in several ways over last school year.

They went from getting 70 percent of the Ds and Fs in English and math to 60 percent. In 2016-17, 46 percent of boys compared with 55 percent of girls were on track, meaning they earned a C or higher in reading and math and had an attendance rate of at least 95 percent. In 2017-18, the percentage of boys on track increased by 23 percentage points compared to 19 points for girls. But sitting in her office at Fuller one day earlier this summer, McCottrell admitted something about her efforts for black boys.

“Nothing is solved,” she said.

Despite some progress last school year, when the 2018-19 school year stars, black boys at Fuller will still lag behind black girls. Forces outside of education like poverty, mass incarceration, and racial discrimination will continue to disadvantage black youth in ways that manifest in classrooms, where they land heaviest on black boys.

The odds aren’t yet even for black boys at McCottrell’s school, or at most schools across America. However, Mccottrell believes that educators learned a lot that they can build on down the line.

Next year, McCottrell said she’s urging teachers to incorporate more of the black experience and black voices into lesson plans and to increase small-group instruction.

Teachers are having more data conferences with McCottrell and with each other to guide instruction and target specific students’ needs. McCottrell is also promoting more social-emotional learning techniques and restorative practices rather than punitive approaches to discipline.

Marlene Aponte, AUSL’s director of coaching,  said that in some ways Fuller’s story resembles other schools’ in the years after turnarounds. After focusing on rigorous instruction and ambitious growth targets,“we’re starting to really hone in on some of the pieces that we may have overlooked, such as gender bias, gender equity, access in equity,” she said.

McCottrell wants her boys to have the tools to succeed. She knows there are some issues that her school won’t be able to solve.

But it’s a start.

testing accountability

Pressuring schools to raise test scores got diminishing returns, new study of No Child Left Behind finds

PHOTO: Ann Schimke

Does tightening the screws on schools and teachers lead to benefits for students?

For the past couple of decades, school reform efforts have assumed that the answer is yes. Setting ambitious goals, and putting pressure on schools to reach them, would push students ahead. And past research has shown that math scores rose as more states began threatening and sanctioning schools with low test scores in the 2000s.

But a new study shows that continuing to to “raise the bar” during the No Child Left Behind era only had a modest effect at best. That raises questions about whether the small gains were worth the political controversy, and what critics claim were the educational costs, of putting a greater focus on test scores.

“These results suggest that the ratcheting [up] of test-based accountability pressures alone is not enough to sustain improvements in student achievement,” conclude researchers Vivian Wong, Coady Wing, David Martin, and Anandita Krishnamachari.

Their paper, which has not been formally peer-reviewed, focuses on the several years after the federal No Child Left Behind law was signed in 2002. The law — which passed with bipartisan support but would eventually draw bipartisan ire — required states to test students annually and set goals for schools. Schools that didn’t meet them faced sanctions.

States each set their own targets using different tests. But the researchers attempted to ask the same questions of each state: How hard was it for each school to hit its goals, and how did that change between 2003 and 2011? Then, they looked at how students did on the National Assessment for Educational Progress. Did states see larger gains on the federal low-stakes test after making life tougher on schools?

In many states, it really did become harder and harder for schools to measure up. In 2008, Education Week noted that California’s school failure rate jumped from 34 to 48 percent between 2007 and 2008. In Vermont, the climb was even steeper: from 12 percent of schools failing to 37 percent.

This added pressure, the authors conclude, seemed to lead to national gains in eighth grade math and reading. But the effect was tiny: about half a point in both subjects. (For comparison’s sake, the difference in performance between white and black students in eighth grade math was 32 points on the latest test.)

“Though they find positive effects, like everyone in this literature, they are small [effects],” Tom Dee, a Stanford education professor.

That said, the gains were largest for certain disadvantaged groups: English language learners, Hispanic students, students with disabilities, and students who started at the lowest levels of performance.

There was no evidence of higher standards causing any improvements in fourth grade math or reading.

If this shows that raising the bar doesn’t do much, though, past research has shown that just having a bar can make a big difference.

In states that didn’t have accountability systems at all before No Child Left Behind, creating them led to big gains on national low-stakes math tests: 8 points in fourth grade and 5 points in eighth grade, according to a study from Dee.

Together, this research bolsters a theory known as the “accountability plateau” — that creating tougher rules boost performance, but ratcheting up the pressure leads to diminishing returns.

“It seems like when you implement an accountability system there’s an initial bump, but after that continued gains are hard to come by,” said Morgan Polikoff, a professor at the University of Southern California who has studied standards and accountability systems.

Dee was more skeptical of this idea. Schools’ goals were getting harder and harder to reach just as criticism of the law was cresting and politicians were considering changes.  

“Districts may have understood it was a nudge and a wink and it didn’t really have teeth,” he said of the law.

No Child Left Behind’s replacement, the Every Student Succeeds Act, takes a different tack. Instead of giving each state discretion in how many schools are identified as failing and requiring them to ramp up the consequences over time, the law requires each state to identify 5 percent of schools as low-performing.

The latest study suggests that might be a preferable approach if states are able to figure out better ways to help a small group of struggling schools improve. Turnaround efforts — including a prominent federal program backed by a lot of money — have often produced disappointing results.

“It remains unclear how states will implement ESSA,” write the researchers. “But the federal law will likely not succeed if performance requirements are not accompanied by additional support for educators.”