value added?

Eight years ago, the L.A. Times published teachers’ ratings. New research tells us what happened next.

In 2010, the Los Angeles Times did something that hadn’t been done before.

The newspaper published test-score data for thousands of the city’s public school teachers, assigning them a rating based on how they influenced students’ results.

It caused a firestorm. Critics, including many teachers, railed against the measures as misleading and poorly constructed, warning that the data would demoralize teachers. The L.A. Times itself defended the release as necessary transparency.

In an accompanying story, one local teacher suggested it might also help children by empowering parents “to demand a good teacher.”

New research suggests that’s what happened next — but only for certain families.

Publishing the scores meant already high-achieving students were assigned to the classrooms of higher-rated teachers the next year, the study found. That could be because affluent or well-connected parents were able to pull strings to get their kids assigned to those top teachers, or because those teachers pushed to teach the highest-scoring students.

In other words, the academically rich got even richer — an unintended consequence of what could be considered a journalistic experiment in school reform.

“You shine a light on people who are underperforming and the hope is they improve,” said Jonah Rockoff, a professor at Columbia University who has studied these “value-added” measures. “But when you increase transparency, you may actually exacerbate inequality.”

That analysis is one of a number of studies to examine the lasting effects of the L.A. Times’ decision to publish those ratings eight years ago. Together, the results offer a new way of understanding a significant moment in the national debate over how to improve education, when bad teachers were seen as a central problem and more rigorous evaluations as a key solution.

The latest study, by Peter Bergman and Matthew Hill and published last month in the peer-reviewed journal Economics of Education Review, found that the publication of the ratings caused a one-year spike in teacher turnover. That’s not entirely surprising, considering many teachers felt attacked by the public airing of their ratings.

“Guilty as charged,” wrote one teacher with a low rating. “I am proud to be ‘less effective’ than some of my peers because I chose to teach to the emotional and academic needs of my students. In the future it seems I am being asked to put my public image first.”

But a separate study, by Nolan Pope at the University of Maryland, finds the publication of the ratings may have had some positive effects on students, perhaps by encouraging schools to better support struggling teachers.

Pope’s research showed that Los Angeles teachers’ performance, as measured by their value-added scores, improved after their scores were published. The effects were biggest for the teachers whose initial scores were lowest, and there was no evidence that the improvement was due to “teaching to the test.”

“These results suggest the public release of teacher ratings could raise the performance of low-rated teachers,” Pope concluded.

The two studies offer divergent pictures of the consequences of L.A. Times’ move. Pope did not find that higher-scoring students moved into the classrooms of higher-scoring teachers, while Bergman and Hill didn’t find clear evidence that teachers improved.

Those varying results are not entirely surprising, since the researchers used different methods. Pope’s research compared the same teachers before and after their value-added scores were published. Bergman and Hill took advantage of the fact that the L.A. Times only published scores for teachers who taught 60 or more students between 2003 and 2009, creating a natural experiment. The researchers then compared teachers who had taught just more than 60 kids to those teachers who had taught just under 60.

Rockoff of Columbia said he found both studies credible.

A third study, published in 2016, looks at an entirely different question: Did housing prices in Los Angeles increase near schools with more highly rated teachers?

Not really, according to the paper. That’s somewhat surprising, because past research has shown that housing zoned for schools with higher overall test scores and ratings is more expensive.

The researchers suggest that that might be because families had a hard time understanding what the ratings represented, and that some may have tuned out because of the surrounding controversy.

The results come in a very different political climate than around the time of the public release of the scores, when conversations about teacher performance had reached a fever pitch.

In 2010, then-Secretary of Education Arne Duncan praised the publication of teacher ratings. He used federal carrots and sticks to encourage states to use student test scores as part of how teachers are judged, a policy most states adopted.

But since then, states like New York and Virginia have barred the public release of this performance data, while media organizations have increasingly shied away from publicizing them. The new teacher evaluation systems have run into political challenges, and in some cases not had the hoped-for effects on student performance. And the federal education law passed in 2016 specifically banned the secretary of education from pushing teacher evaluation rules.

negotiations

Aurora school board reverses course, accepts finding that district should have negotiated bonuses with union

Students in a math class at Aurora Central High School in April 2017. (Photo by Yesenia Robles, Chalkbeat)

Following weeks of criticism, the Aurora school board on Tuesday reversed course and accepted an arbitrator’s finding that a pilot bonus system violated the district’s agreement with the teachers union.

The Aurora school district rolled out an experiment last year to offer bonuses to some teachers and other staff in hard-to-fill positions, such as psychologists, nurses and speech language pathologists.

The teachers union argued that the plan should have been negotiated first. An arbitrator agreed and issued a report recommending that the pilot program stop immediately and that the district negotiate any future offerings. The union and school board are set to start negotiations next month about how to change teacher pay, using new money voters approved in November.

When school board members first considered the arbitrator’s report last month, they declined to accept the findings, which were not binding. That raised concerns for union members that the district might implement bonuses again without first negotiating them.

Tuesday’s new resolution, approved on a 5-1 vote, accepted the full arbitrator’s report and its recommendations. Board member Monica Colbert voted against the motion, and board member Kevin Cox was absent.

Back in January 2018, school board members approved a budget amendment that included $1.8 million to create the pilot for incentivizing hard-to-fill positions. On Tuesday, board member Cathy Wildman said she thought through the budget vote, the school board may have allowed the district to create that incentive program, even though the board now accepts the finding that they should have worked with union before trying this experiment.

“It was a board decision at that time to spend that amount on hard-to-fill positions,” Wildman said.

Board president Marques Ivey said he was not initially convinced by the arbitrator’s position, but said that he later read more and felt he could change his vote based on having more information.

Last month, the Aurora school board discussed the report with its attorney in a closed-door executive session. When the board met in public afterward, it chose not to uphold the entire report, saying that the board could not “come to an agreement.” Instead board members voted on a resolution that asked the school district to negotiate any future “long-term” incentive programs.

Union president Bruce Wilcox called the resolution “poorly worded” and slammed the board for not having the discussion in public, calling it a “backroom deal.” Several other teachers also spoke to the board earlier this month, reminding the newest board members’ of their campaign promises to increase transparency.

Board members responded by saying that they did not hold an official vote; rather the board was only deciding how to proceed in public. Colorado law prohibits schools boards from taking positions, or votes, in private.

The board on Tuesday also pushed the district to provide more detailed information about the results of the pilot and survey results that tried to quantify how it affected teachers deciding to work in Aurora.



story slam

The state of teacher pay in Indiana: Hear true stories told by local educators

It’s time to hear directly from educators about the state of teacher pay in Indiana.

Join us for another Teacher Story Slam, co-hosted by the Eiteljorg Museum of American Indians and Western Art, Chalkbeat Indiana, and Teachers Lounge Indy. Teacher salaries are the hot topic in education these days, in Indiana and across the country. Hear from Indianapolis-area teachers who will tell true stories about how they live on a teacher’s salary.

Over the past two years, Chalkbeat has brought readers personal stories from the teachers, students, and leaders of Indianapolis through our occasional series, What’s Your Education Story? Some of our favorites were told live during teacher story slams hosted by Teachers Lounge Indy.

Those stories include one teacher’s brutally honest reflection on the first year of teaching and another teacher’s uphill battle to win the trust of her most skeptical student.

Event details

The event will be held from 6-8 p.m. on Friday, March 15, at Clowes Court at the Eiteljorg, 500 W Washington St. in Indianapolis. It is free and open to the public — please RSVP.

More in What's Your Education Story?