value added?

Eight years ago, the L.A. Times published teachers’ ratings. New research tells us what happened next.

In 2010, the Los Angeles Times did something that hadn’t been done before.

The newspaper published test-score data for thousands of the city’s public school teachers, assigning them a rating based on how they influenced students’ results.

It caused a firestorm. Critics, including many teachers, railed against the measures as misleading and poorly constructed, warning that the data would demoralize teachers. The L.A. Times itself defended the release as necessary transparency.

In an accompanying story, one local teacher suggested it might also help children by empowering parents “to demand a good teacher.”

New research suggests that’s what happened next — but only for certain families.

Publishing the scores meant already high-achieving students were assigned to the classrooms of higher-rated teachers the next year, the study found. That could be because affluent or well-connected parents were able to pull strings to get their kids assigned to those top teachers, or because those teachers pushed to teach the highest-scoring students.

In other words, the academically rich got even richer — an unintended consequence of what could be considered a journalistic experiment in school reform.

“You shine a light on people who are underperforming and the hope is they improve,” said Jonah Rockoff, a professor at Columbia University who has studied these “value-added” measures. “But when you increase transparency, you may actually exacerbate inequality.”

That analysis is one of a number of studies to examine the lasting effects of the L.A. Times’ decision to publish those ratings eight years ago. Together, the results offer a new way of understanding a significant moment in the national debate over how to improve education, when bad teachers were seen as a central problem and more rigorous evaluations as a key solution.

The latest study, by Peter Bergman and Matthew Hill and published last month in the peer-reviewed journal Economics of Education Review, found that the publication of the ratings caused a one-year spike in teacher turnover. That’s not entirely surprising, considering many teachers felt attacked by the public airing of their ratings.

“Guilty as charged,” wrote one teacher with a low rating. “I am proud to be ‘less effective’ than some of my peers because I chose to teach to the emotional and academic needs of my students. In the future it seems I am being asked to put my public image first.”

But a separate study, by Nolan Pope at the University of Maryland, finds the publication of the ratings may have had some positive effects on students, perhaps by encouraging schools to better support struggling teachers.

Pope’s research showed that Los Angeles teachers’ performance, as measured by their value-added scores, improved after their scores were published. The effects were biggest for the teachers whose initial scores were lowest, and there was no evidence that the improvement was due to “teaching to the test.”

“These results suggest the public release of teacher ratings could raise the performance of low-rated teachers,” Pope concluded.

The two studies offer divergent pictures of the consequences of L.A. Times’ move. Pope did not find that higher-scoring students moved into the classrooms of higher-scoring teachers, while Bergman and Hill didn’t find clear evidence that teachers improved.

Those varying results are not entirely surprising, since the researchers used different methods. Pope’s research compared the same teachers before and after their value-added scores were published. Bergman and Hill took advantage of the fact that the L.A. Times only published scores for teachers who taught 60 or more students between 2003 and 2009, creating a natural experiment. The researchers then compared teachers who had taught just more than 60 kids to those teachers who had taught just under 60.

Rockoff of Columbia said he found both studies credible.

A third study, published in 2016, looks at an entirely different question: Did housing prices in Los Angeles increase near schools with more highly rated teachers?

Not really, according to the paper. That’s somewhat surprising, because past research has shown that housing zoned for schools with higher overall test scores and ratings is more expensive.

The researchers suggest that that might be because families had a hard time understanding what the ratings represented, and that some may have tuned out because of the surrounding controversy.

The results come in a very different political climate than around the time of the public release of the scores, when conversations about teacher performance had reached a fever pitch.

In 2010, then-Secretary of Education Arne Duncan praised the publication of teacher ratings. He used federal carrots and sticks to encourage states to use student test scores as part of how teachers are judged, a policy most states adopted.

But since then, states like New York and Virginia have barred the public release of this performance data, while media organizations have increasingly shied away from publicizing them. The new teacher evaluation systems have run into political challenges, and in some cases not had the hoped-for effects on student performance. And the federal education law passed in 2016 specifically banned the secretary of education from pushing teacher evaluation rules.

positive discipline

How this Indiana district is rethinking discipline to keep kids in school

PHOTO: Alan Petersime

Warren Township administrators are taking a new approach to discipline — often forgoing traditional punishments, such as suspensions, in favor of interventions that better support the children who have gotten into trouble.

It’s called “positive discipline,” and it takes into account that traumatic events, such as a parent in jail, the loss of a family member, or homelessness, may be at the root of a child’s misbehavior. In those cases, experts say making a home visit or providing mentoring — paired with a consequence such as detention — can be more beneficial than forcing a student, who may need help, out of school.

“Children have problems at school because of things that have happened in their background,” JauNae Hanger, president of the Children’s Policy and Law Initiative of Indiana. said. “As educators we’ve got to be cognizant of that.”

The result of these more mindful discipline policies: better care for students, and fewer students missing class.

This softer approach to discipline is gaining traction throughout the United States, particularly as schools confront high suspension and expulsion rates that have been found to unfairly target students of color.

Indiana in particular has grappled with such disproportionately harsh discipline for black students. Now, the state is establishing guidelines for educators to use this new approach in their own classrooms.

“Even one detention in the ninth grade can increase the risk of a child going into the juvenile justice system,” Hanger said. “School discipline data suggests that we do have a problem. It’s a statewide problem.”

Through a new state law passed this year, the Indiana Department of Education is developing a best practice model for districts like Warren Township that are interested in implementing “positive discipline,” which seeks to teach rather than to punish.

The model will provide ways to reduce out-of-school suspension and inequities in discipline, to limit referrals to law enforcement or arrests on school grounds, and to draft or strengthen policies that address issues of bullying on school property.

Schools are not required to adopt the best practice model, nor are they required to reduce their suspensions and expulsions, but the state must provide information and support to districts upon request.

“Operating in the mode of punishment without supports is not productive for families or kids,” said James Taylor, Warren Township’s director of student services. “We want to support families as much as we can because we’re the first line of defense before the juvenile system.”

Warren Township is one of seven Indiana districts that participated in specialized training last year to learn how to respond to misbehavior in a way that considers what’s happening in a student’s life outside of the classroom. Children in poverty are more likely than their peers to experience traumatic events.

The training is hosted by the Children’s Policy and Law Initiative of Indiana, which is known for its efforts around reforming laws, policies, and practices to keep children in school and out of the criminal justice system. Training comprises a two-day summit and four one-day sessions during the course of a year.

Each session is designed for a different group of school employees, including administrators, teachers, and school resource officers. The final session brings everyone together for cross-disciplinary training, team building, and strategic planning.

Jim Sporleder is a trauma-informed coach who helps lead positive discipline training sessions across the U.S., including Indiana. He has seen how adopting a different mindset on discipline can be difficult: “It’s going against tradition,” Sporleder said. “It’s going against how we were raised.”

That was an early challenge for Warren Township schools. The strategy requires “a paradigm shift,” Taylor said.

“Everybody has to be on board, but everybody’s not always on board,” he said. “It’s tough to get people out of that mindset that this kid really painted a black eye for our school, and how do we move past the pain and the hurt of the situation when it happens?”

At Warren Central High School, 584 out of 3,710 students received out-of-school suspensions before the district began taking this new approach in 2016-17. But last year, when the district started the positive discipline training, the number of out-of-school suspensions decreased by more than 15 percent, according to state data.

Warren Township now outlines its approach to positive discipline in its student handbook, which defines discipline on the cover as, “instruction that corrects, molds, or perfects character and develops self-control.”

At the Children’s Policy and Law Initiative of Indiana, Hanger said she uses schools’ discipline data to identify schools with a high rate of suspensions and where leaders are willing to address the issue. She and her team are still collecting data, but they believe suspensions will go down within the first year of training.

Suspensions have long been a problem in Indiana: During the 2012-13 school year, one in 10 Indiana students were suspended. For black students, the number was even greater — one in five.

However, some groups that represent educators have had concerns about how to roll out this approach effectively. Tim McRoberts, associate executive director of the Indiana Association of School Principals, said he doesn’t want schools to restrict how teachers can discipline students. Teachers still need to have a full range of options when dealing with incidents.

Still, he said he supports keeping students in school.

Sporleder, the trainer, said that in order for a shift toward positive discipline to work, the entire school has to change the way it looks at all students.

“Some people separate out who’s trauma and who’s not,” Sporleder said. “We can’t do that because we don’t know. Your most compliant student in classroom could be most traumatized. Trauma isn’t a checklist. It’s who you are.”

Asked and answered

Longtime advocate (and current First Lady) Diana Rauner sizes up the challenges ahead for early education in Illinois

PHOTO: Courtesy of Ounce of Prevention

Fluent in the languages of developmental psychology (her Ph.D., from the University of Chicago) and finance (her MBA, from Stanford University), Illinois First Lady Diana Rauner is equipped more than most to navigate the maze that is early childhood education in America. As anyone who has tried to find, or build, a quality program for a child under 5 knows, there are plenty of hurdles to securing good options: availability and affordability; too few full-day seats for families that require them; and low pay and turnover among providers, just to name a few.

In her 11 years in leadership of the Chicago advocacy group Ounce of Prevention, Rauner has used her platform to push for higher quality programs for the youngest children in Illinois and nationwide. As the wife of Illinois Gov. Bruce Rauner (who is up for election this fall against another early childhood education advocate, businessman J.B. Pritzker, who has funded some of Ounce of Prevention’s work), she’s as fluent in political speak as she is everything else: “Early childhood is not a partisan issue,” she says.

She spoke to Chalkbeat Chicago’s Cassie Walker Burke amidst big developments for early childhood advocates: Mayor Rahm Emanuel is touting the rollout of an ambitious universal pre-K program in Chicago, and the state is set to release a trove of data on kindergarten readiness later this month. She was joined in parts of the conversation by Ireta Gasner, vice president of Illinois policy for Ounce of Prevention.

The interview has been condensed and edited for publication. 

Why is the issue of early childhood education so important to you?

DIANA RAUNER: This is the most important human capital and social justice issue for our nation. We are increasingly a society that requires everyone to have both social-emotional self-regulation skills and the flexibility to continue to learn throughout your life. We know that, as Warren Buffett says, all men are created equal and that lasts for the first 15 seconds. Something we know now that we didn’t know a generation ago, or even 10 years ago, is the importance of the prenatal period to long-term health and health outcomes. And so, actually it’s not even true that all people are born equal.

We really have to ensure that we’re giving all families the kinds of supports they need in order to help their children develop to their highest potential. It’s a moral issue, but it’s also an economic and civic issue as well.

Many people understand the value of early childhood programs, but we are also seeing the percentages of students enrolled in public pre-K in Illinois and Chicago dropping. What is your assessment of why this is happening?

RAUNER: The Early Childhood Block Grant (which is the Illinois State Board of Education’s early childhood education program) was cut several years in a row starting in 2010. We lost $80 million over a number of years. The block grant didn’t begin to grow again until 2016, so because of that, enrollment did drop during that time. It has rebounded. We are not sure that the numbers are not back up and over, but the (latest publicly available counts) aren’t current. We just aren’t sure.

IRETA GASNER: The other thing impacting that is that Illinois for decades had one of the nation’s shortest pre-K days: We were serving kids in 2½ -hour programs. Knowing what we’ve learned about at-risk kids, (we need to) serve up a longer day. But there’s some tension there: We can give a lot of kids a little dosage or we can grow the programs back up a little more slowly and give kids who need it the right dosage. So there’s some nuance around the number of kids served now compared to where we were 10 years ago or so.

Funding for early childhood comes from a lot of places; it’s complicated. Some providers lost a state grant recently and complained the application process rewarded good grant writing, not quality programming. How do we build a pipeline of providers and sustain it?

RAUNER: Those are huge questions. Clearly, we know that the Early Childhood Block Grant, while we’ve seen increases, is still not sufficient to serve all the kids who need it. There is a big gap between how we’re funding now and where we need to be to reach all the kids who need it. The recompetition (to reapply for the Preschool for All grants that fund programs throughout the state) that was done this year wasn’t a perfect process. You combine the fact that the process wasn’t perfect with the fact that there’s not enough money to go around, and you end up with an outcome that doesn’t satisfy everyone.

You talked about what it takes to do quality. As a state, for a very long time, we’ve had probably one of the strongest early childhood systems — birth to 5 — of any state. It’s truly a birth to 5 system, it has a birth to 3 set-aside, and it has real attention to birth to 3 funding. It’s also a mixed-delivery system, which means both community-based settings and school settings. It’s meant to meet families where they are and meet families’ different needs. And it sets aside investments for quality infrastructure: that means professional development, data collection, research, innovative programming. That’s been a hallmark of the way that we in the state, and in the city, have prioritized our early childhood system.

Researchers also stress that only quality programs really move the needle with kids from low-income backgrounds. How do we ensure quality?

RAUNER: For a long time in education we’ve had this myth that teachers are somehow superhuman, and that a great teacher is great, and that an average teacher is average, and that bad teachers are bad. At Ounce, what we’ve focused a lot on this: Teachers, like other adults, work in organizations. They work in organizations that either support and enhance and develop their performance, or they work in organizations that don’t. Rather than focusing everything on the individual teacher and how good or bad the individual teacher is, we need to look at the organizational supports that help that teacher do great work in the classroom. We focus on leadership in instructional support and instructional excellence, and all of the essential elements of organizational support for great teaching — it doesn’t happen in a vacuum.

What’s an example of a program that you’ve visited recently that you’d like to see scaled?

RAUNER:  Our work with universal newborn support. The Illinois Family Connects program is a model that comes out of Durham, North Carolina. It is a nurse home visiting program that serves as a coordinated intake to a web of community supports for new parents and their children. It is a validated and well-trained individual assessment of a new parent and new family and a system of referrals that support families in all kinds of things they need. It’s a universal system, and it’s one that right now we are piloting in two Illinois counties, Peoria and Stephenson. We are hoping to expand in the city of Chicago and other counties across the state.

A year in, what have you learned from the newborn visitation program?

RAUNER: First of all, we’ve learned it’s very, very welcome by families. The uptake rates are high: More than 80 percent of families say, yes, I’d love to have a nurse visit me at home. We’ve also learned that 97 percent of the families that have visits have some kind of need. Many of those needs, more than half, can be addressed right in that visit. Over a third, though, are getting referrals to some kinds of support, including mental health programs, intensive home visiting programs, other kinds of family support programs. A very small percentage, maybe around 1 percent, are getting referred for really urgent issues: medical issues, mental health issues, safety issues in their home. That’s amazingly important and critical to getting addressed quickly.

Do you think Illinois has the ability to scale a program like that? Is there the will on the public side and on the political side?

RAUNER: I think the most important thing we can do is demonstrate both the important outcomes we are seeing, and also, over time, population-level changes that save the state money. In Durham, this program has shown substantial decreases in emergency room visits and child welfare referrals. Those are extremely expensive. And so, if you can really do this kind of program at scale and prevent those kinds of expenditures, you can actually save money and put that money toward prevention and toward serving more families.

"For a long time in education we’ve had this myth that teachers are somehow superhuman, and that a great teacher is great, an average teacher is average, and bad teachers are bad. "Diana Rauner

What else can Chicago learn from other states and cities that are doing early childhood well?

RAUNER: We’ve worked really hard to bring things back here. We brought a program that was developed in Florida called Baby Court to Illinois, which is intensive supports for families with very young children in the child welfare system.We’re always thinking about opportunities to bring innovations here and try new things, and we’re also trying to share some of our innovations with other states.

Pay is really low throughout the early childhood education system. Do you see a realistic path to changing that?

RAUNER: There’s no silver bullet unless we have $100 billion drop from the sky. But there are a lot of things we’ve been doing in Illinois: alternative certification programs, pathway programs, better connections between our high schools and community colleges. There are a lot of things we want to try, but it will take ongoing attention. It’s a little bit like — oh, I shouldn’t make a sports analogy — but you gain a yard at a time. You just keep pushing. It’s not like something is going to transform overnight.

As Chicago has more CPS-backed pre-K seats come online and pays teachers on the CPS scale, community programs are going to have to compete with their wages. Isn’t that going to become an issue here as universal pre-K rolls out?

RAUNER: It is an issue, and part of that is ensuring that the universal pre-K program is still a mixed-delivery system (a mixed-delivery system includes community providers who receive public funding as well as school district-funded programs). We want to make sure the community providers are still part of the preschool system. Clearly pay parity is really important in the long run, and we’re certainly still some way away from that across the board, but it is a really high priority.

It’s important to recognize that a birth-to-3 program is more expensive than a preschool, so that’s one of the reasons why, as universal preschool rolls out, we’ll be focused on it continuing to become a community-based program.

Why is a mixed-delivery system something you advocate?

RAUNER: Very often, community-based programs are actually more reflective and responsive to the communities they serve. Another reality: Many community-based programs blend and braid funding streams, so they can provide full-day coverage for children whose parents are working. The trouble is that programs that operate 6 hours a day cannot serve families who need 10 hours of coverage.

Will the universal pre-K program being rolled out in Chicago have a watershed effect on other places in the state?

RAUNER: I don’t know. We’ll have to see. We have many districts in the state, so they all have very different priorities.

Later this month, the state will release its first kindergarten readiness reports. What can we expect?

RAUNER: It’s a milestone. It’s a tool that serves many different purposes. One is a professional development tool for kindergarten teachers, it helps them see and observe their students and understand (what it takes) to move them along on their developmental path. It’s helpful for parents to understand the range of developmental expectations for kindergarten. And it’s helpful for policy makers to understand how we’re doing relative to our expectations and goals.

I do think that we have to be prepared for some instability in the data in the very beginning — that’s pretty typical. But obviously, well — sometimes the truth is a difficult thing. Here we have always focused on third-grade test scores as evidence of the achievement gap, but we all know the achievement gap opens up much, much earlier. Being able to articulate that and identify that and document it — we hope it will change the conversation so we can talk about it at a much earlier level.

"Sometimes the truth is a difficult thing."Diana Rauner

I imagine that wasn’t an easy sell. How did you convince people?

RAUNER: We had many conversations. We did this as an early childhood and K-12 partnership, we brought together teachers, and school administrators, and early childhood advocates, and researchers. We wanted to make sure this tool was developmentally appropriate, that it was valid and reliable, and that it really served the purpose of professional development as well as accountability. Not accountability for individual students — we wanted to be really sure it wasn’t used in any inappropropriate ways to penalize individual students — but rather, as a way to hold us adults accountable to our very youngest learners.

We’re in the middle of a heated governor’s race, as you know. How does this affect the work you do?

RAUNER: We know from our work in Illinois and at the federal level that early childhood is not a partisan issue. The majority of every partisan persuasion are strongly in favor of investment in early childhood. That makes our work easy. It means we are able to articulate a vision for early childhood education and to promote the best practices to all candidates and all legislators. We have seen strong bipartisan support here in Illinois on the issue for decades.