unlikely friends

Conservative think tank finds ‘meaningful’ academic progress at New York City’s Renewal schools

PHOTO: Patrick Wall
Mayor Bill de Blasio at Brooklyn Generation School — part of the Renewal program

Mayor Bill de Blasio’s big bet on pumping millions of dollars into low-performing schools instead of closing them down is creating “meaningful” academic benefits.

That’s according to a forthcoming report from an unlikely source: the conservative-leaning Manhattan Institute, a frequent critic of the mayor’s education policies. The report is the first independent analysis to show that the Renewal program is producing academic benefits, as measured by math and reading scores on state tests.

School Renewal — a program that infused 94 of the city’s lowest-performing schools with additional social services, nonprofit partnerships and academic support — tended to produce larger academic gains than the rest of the city’s schools, including those that had similarly low test scores before the program began.

In all, the report estimates, Renewal boosted student achievement by a “meaningful magnitude” — the equivalent of about 49 days of extra instruction in reading and 33 days in math, as measured by gains in state test scores. (The report focuses only on elementary and middle schools. It does not look at academic progress in Renewal high schools, which comprise a little more than a third of the program.)

“The evidence I find is that that the schools are better than they would have been without the label,” said Marcus Winters, the study’s author.

That finding is good news for an administration that has struggled to point to rigorous statistical evidence that its plan for low-performing schools is creating clear progress. It also comes at a key moment, as the program is nearing the end of the third year of what de Blasio initially described as a three-year program (though it is slated to continue next year).

Still, the Manhattan Institute’s relatively positive findings are complicated by a recent analysis conducted by Teachers College professor Aaron Pallas. Using a different statistical model and an additional year of data, Pallas found that Renewal schools generally did not make bigger gains in reading or math when paired with other struggling schools that didn’t receive extra support.

“There’s some hint of progress in raising test scores that wasn’t present in my analysis,” Pallas said, noting the Manhattan Institute analysis focused on one year of data. “It’s not, I think, a strong case.”

In its response to the report, the city seemed reluctant to seize on its specific findings as evidence of the program’s success, pointing instead to general increases in achievement at Renewal schools. “Renewal Schools are making real progress,” education department spokesman Michael Aciman said in a statement. “Across the program, graduation rates are up, chronic absenteeism is down, state test scores are improving, and teachers are developing stronger instructional practices.”

To assess the program, Winters used a model that measured yearly changes in test scores in 2015 and 2016 at Renewal schools compared with those outside the program. The Renewal program had the strongest effect in 2016, the year after it was fully implemented, boosting math and reading scores by a statistically significant margin overall (though the positive benefits were not consistent across all grade levels).

The positive effect becomes less pronounced when Renewal schools are compared to others that also had low scores, or when the timeline is extended to include 2015 data from when the program was still rolling out.

Jonah Rockoff, an education researcher at Columbia University who reviewed the Manhattan Institute report and Pallas’s data, cautioned against claiming the new report offers definitive evidence of the Renewal program’s effect on test scores.

“The [Manhattan Institute] report is more upbeat on Renewal schools raising test scores,” he said. “My sense is that any positive effects are quite small.”

Winters also claims that the Renewal program — which has cost roughly $386 million so far — is both more costly and less effective than former Mayor Michael Bloomberg’s approach of giving schools letter grades and closing ineffective ones.

Under Bloomberg’s letter grade system, for instance, schools that received an ‘F’ improved the following year by the equivalent of 36 days of learning in reading and 72 days in math, the report says, more than schools improved under Renewal. But the study does not include a rigorous cost-benefit analysis of Bloomberg’s versus de Blasio’s efforts to improve struggling schools — and city officials said the report’s comparisons are unfair.

“The A-F grading system does not take into account the systemic challenges that many Renewal schools faced when they entered the program,” Aciman said. “The study overestimates the impact that a single letter grade can have on improving a school, and underestimates the impact of the research-based supports that are being implemented at Renewal schools to improve classroom instruction, school climate and student performance.”

Correction: Due to errors in the original Manhattan Institute report, this story has been corrected to reflect the accurate number of extra days of instruction created by the Renewal program and Bloomberg’s accountability system.

legal showdown

Lawsuit targets New York City program that strands poor students without required special ed services

PHOTO: Alex Zimmerman
Public Advocate Letitia James announced a report earlier this month criticizing the city's special education voucher program.

A program that makes New York City parents responsible for finding their own special education services — but that often leaves them with no services at all — is under legal attack.

The class action lawsuit, filed Thursday in a federal district court, aims to reform the city’s process for ensuring that students with disabilities receive “related services” — which include physical therapy, certain medical services and counseling, among other therapies.

When the city’s education department is unable to offer those services itself, or through a contractor, parents are given a voucher that can be used to pay an outside provider. But that system puts the onus on families to find providers, and about half of the 9,164 vouchers issued during the 2015-16 school year went unused, according to a report issued earlier this month by the public advocate’s office.

The lawsuit centers on the Bronx, where the problem is particularly acute. In District 8, which includes Hunts Point, Throgs Neck and Soundview, 91 percent of the 129 vouchers issued last school year went unused — the highest rate anywhere in the city.

The city’s public advocate found that families face a number of barriers to using the vouchers: They often struggle to find providers in their neighborhoods, have difficulty arranging for transportation and getting reimbursed to send their children elsewhere, or simply can’t find providers who are responsive.

In part because of those challenges, an attorney who helped bring the lawsuit said the city can’t simply offer a voucher to fulfil its obligation to provide special education services.

“The DOE has to ensure that students actually get [services]” said Seth Packrone, a lawyer at Disability Rights Advocates, which contributed to the public advocate’s report. “They can’t just issue a voucher and then step away.”

The goal of the litigation is to force the education department to come up with a plan to ensure that students in the Bronx receive the services they have been guaranteed, Packrone said. It is not yet clear what that plan could entail or how it could affect other neighborhoods, which also have large numbers of unused vouchers.

The complaint says the city’s voucher program violates multiple federal laws that guarantee students with disabilities a free and appropriate public education. The plaintiffs in the case are two Bronx students and Bronx Independent Living Services, a nonprofit that works with students who have disabilities.

Education department spokeswoman Toya Holness wrote in a statement: “We are dedicated to meeting the needs of students with disabilities and in the small percentage of cases when we issue a related service authorization, we work with families to connect them with an appropriate provider in their area.”

She referred questions about the lawsuit to a law department spokesman, who said the city is reviewing the complaint.

First Person

Yes, an A at one school may be a C at another. It’s time we address the inequity that got us there

PHOTO: Brett Rawson
Yacine Fall, a student who shared her experience realizing that an A in her school wasn't the same as an A elsewhere.

I was struck by a recent Chalkbeat piece by a young woman who had earned a high GPA at a middle school in Harlem. Believing herself well prepared, she arrived at an elite high school only to find herself having to work hard to stay afloat in her classes.

Her A’s, it seemed, didn’t mean the same thing as the A’s from other, more affluent, schools.

As a teacher, I know that she’s right. Grades are different from school to school, district to district, and I suspect, state to state. And it presents a problem that cannot easily be solved — especially in English, the subject I teach.

The students who sit before us vary greatly. Some schools have students who are mired in poverty and who are also not fluent in English. (Some entire districts are this demographic. I taught in one for many years.) Other schools are quite affluent and have no English language learners. Guess which population demonstrates stronger academic skills?

We teachers cannot help but get normed to our population. We get used to seeing what we always see. Since an A is “excellent,” we tend to give A’s — really, all grades — in relation to the population with which we work. To get an A in any school means that the student is doing an excellent job relative to their peers.

When I taught in my old middle school, most kids arrived below grade level in math and English, and some were several years below. We became so used to seeing below-grade-level work that it became our “normal.” When an eighth-grader who came to us at a third-grade level turned in four or five pretty good paragraphs on a topic, we were elated.

That kid has come so far! We would bring that assignment out at the next department meeting and crow about her success. And we would award an A, because she did an excellent job in relation to her peers.

The trouble is, you take the same assignment down the highway 10 miles to an affluent school, and that same paper would earn a C-minus. Their eighth-graders came to them using strong theses, well developed points, and embedded quotations. To get an A in that school, the student has to do an excellent job relative to much more accomplished peers.

Kids who are just learning English, who are homeless or move frequently, who could be food-insecure, don’t have those skills. They’re not incapable of developing those skills. But they are unlikely to have them yet because of the challenges they face.

I now teach students in a highly competitive magnet program in another state (600 applicants for 150 seats, to give you an idea). Now I am normed so far the other way, it makes me dizzy. These students have skills that I never dreamed any eighth-grader could possess. The eighth-graders I taught this year wrote at a nearly professional level. Many of them score in the 99th percentile nationwide for both math and English.

Now I realize that, in my old district, we almost never saw a truly advanced student. In fact, not only had most of us never seen an advanced paper, we rarely saw any paper that was above partially proficient, even from students we thought were working above grade level.

The reality is that if we truly tried to hold everyone to the same bar, we would see even more troubling patterns emerge.

We would see the good grades going to rich white kids, those who get museums and vacations and Starbucks in the summer, and we would see the failing grades go to the poor kids — entire schools, even districts, full of poor kids who aren’t good with English and who spend their summers in front of the TV while mom and dad work.

So we have these very different sets of standards, even with the Common Core. There is a faction who would say this is “the soft bigotry of low expectations” that George W. Bush talked about. I say this shows that socioeconomic status and students’ home lives are the major predictors of success in school, and that the bigotry that causes that is real.

What does all this mean for the student who wrote the original piece about her transition to high school? What it means for her, immediately, is she sees firsthand the vast differences in preparation and opportunity between the socioeconomic classes. In the long term, it could mean a lot as far as college choices go. I don’t think we know yet how to really solve this problem.

We as a society need to address the factors that limit access and equity for poor and minority children. Leveling that particular playing field may be the most important charge with which educators are tasked.

Mary Nanninga is a middle school English teacher in Montgomery County Public Schools in Maryland. She previously taught in Westminster Public Schools in Westminster, Colorado.