a test of happiness

When teachers are better at raising test scores, their students are less happy, study finds

PHOTO: Denver Post file
Students practice for a standardized test.

Is a good teacher one who makes students enjoy class the most or one who is strict and has high standards? And are those two types even at odds?

new study that tries to quantify this phenomenon finds that on average, teachers who are good at raising test scores are worse at making kids happy in class.

“Teachers who are skilled at improving students’ math achievement may do so in ways that make students less happy or less engaged in class,” writes University of Maryland’s David Blazar in the study, published in the peer-reviewed journal Education Finance and Policy.

The analysis doesn’t suggest that test scores are a poor measure of teacher quality, but does highlight the different ways teachers may be effective.

The research uses data from four school districts across three states between 2010 and 2013; in one year, students were randomly assigned to fourth- and fifth-grade teachers, allowing researchers to study what effect different teachers had on students. Those students were also surveyed about their behavior, self-efficacy, and happiness in class.

A large body of past research has found that teachers have a meaningful impact on student test scores, and a number of more recent studies have found that teachers also impact other measures — sometimes called non-cognitive outcomes — such as behavior and attendance. 

The latest study asks a few big questions.

First: Do teachers have an impact on students’ attitudes and behavior, as measured by student surveys? Here, the answer is convincingly yes, consistent with the emerging research.

Second: Are the statistical estimates — often called value-added  measures — of teacher impacts on test scores and non-cognitive skills accurate? The study examined this by comparing the statistical estimates to the results from from random assignment, and it found that the answer varies. Value-added measures are quite accurate for predicting test scores — an important finding in light of the charged debate on whether to judge teachers by these metrics. But it concludes that the statistical models are often biased for measuring  impact on student attitudes, suggesting that attempting to evaluate teachers in this regard may be misguided.

Finally: Is a teacher’s performance, measured by test scores, similar to performance according to other measures? This question is especially important because it’s key for understanding how to think about teacher quality and how to evaluate it.

The study concludes there was only a weak relationship between test score performance and student behavior and feeling of efficacy in math. But when it came to student happiness, there was a moderate negative association — on average, greater test score gains meant less happy students.

What explains this potentially surprising inverse relationship?

It could be that teachers who were less demanding were more popular because their instruction was less likely to promote learning — but more enjoyable for students. Maybe those teachers just popped in a video on many days; perhaps they never gave homework.

Blazar, for his part, is skeptical of this theory.

“I’m not sure that’s a likely explanation in large part because teachers’ emotional support for students … seems to be really predictive of how happy students are in class,” he said. “Building an emotionally supportive classroom environment is something that educators and researchers have cared about for a long time.”

Another interpretation, then, is that measures of teacher effectiveness based on test scores leave out important dimensions of what makes a good teacher — such as caring for students, something that might show up in happiness surveys.

Blazar emphasizes that while the correlation was negative and statistically significant it was not strong in size, meaning that there were certainly teachers who succeeded in improving both test scores and happiness.

Past research has generally shown that test-based measures capture some, but not all, of the components of effective teaching. Test score results tend to be only modestly related to other measures of performance, like classroom observations or effects on student attendance.

On the other hand, teachers’ impacts on tests have rarely been negatively related to other measures. In fact, there is usually a small positive association, including with regards to student surveys. Moreover, a number of studies have linked teachers’ and schools’ test score impacts to longer-term results, including adult income and college success.

“[Test score value-added] clearly can’t be all about things we don’t care about, such as test prep, if it translates into longer-run outcomes,” Blazar said.

“I think that both are likely important,” he said, referring to test scores and students’ engagement and happiness in class.

“Hopefully we can get to a place where teachers are good at multiple skills,” Blazar said. “Rather than just documenting this pattern, I would want to use this information to say, if you’re good at raising test scores but not as good at engaging students, how can we get you to a place where you can do both at the same time?”

Want more Chalkbeat? Sign up for our new weekly national newsletter here.

resentment and hurt

‘We are all educators:’ How the teachers strike opened at a rift at one Denver middle school network that will take time to close

PHOTO: Melanie Asmar/Chalkbeat
Students at Kepner Beacon Middle School work on an assignment.

For the first time since this week’s Denver teacher strike exposed divisions in their ranks, the 100 adults who make the Beacon middle school network run gathered in the same room.

Teachers, some still wearing red for the union cause, came with breakfast burritos to share. Upbeat soul music pumped through the speakers, an attempt to set a positive tone.  

Speaking to the group assembled Friday for a long-scheduled planning day in the cafeteria of Grant Beacon Middle School, Executive Principal Alex Magaña opened by acknowledging the awkwardness that had taken a toll on a school community that prides itself on a strong culture.

Some teachers who had gone on strike — exhausted by the experience and exhilarated by the outcome — felt snubbed. Where was the celebration of what they had just fought for?

School administrators were smarting for another reason: A large number of teachers did not return to work on Thursday after the tentative pact was signed, making for another hard day.

Just as it was starting, the effort to heal the Beacon school community was stumbling a bit.

One day after the end of the three-day strike over teacher pay, Denver students had a day off Friday, giving leaders at the 147 district-managed schools affected by the strike the opportunity to begin repairing any damage done. The district administration shared resources with schools, too, including a “lessons learned” tipsheet from the recent Los Angeles teachers strike.

The challenge is proving unexpectedly daunting at the city’s two “Beacon” schools — Grant Beacon in east Denver and Kepner Beacon in southwest Denver — which share a common central administrative staff, approach, and mission to serve the city’s neediest students.

“It’s never been administration-versus-teachers, district-versus-teachers, in the culture we have created here,” said Magaña, who oversees the two schools. “We have a lot of good leadership, a lot of input from teachers. But this caught everyone kind of surprise.”

By “this,” Magaña means the tension that developed on the two campuses before, during, and after the strike that put Denver in an unfamiliar national glare. The 93,000-student district is better known for its unique brand of at times controversial education reform — of which the Beacon network is part — than it is labor strife and division in the educator ranks.

As it became evident that the teachers union was intent on striking, Magaña said he sent a message to the teachers, staff, and administrators on the two campuses.

“I called it out two weeks ago: Be careful with what you say, because it’s going to cause harm and impact our culture,” he said. “Everyone has their own right to make their own individual decision. Respect it. And people were trying to respect it.”

From Magaña’s perspective, it didn’t always happen. At Kepner Beacon, where 96 percent of students qualify for subsidized lunches, the young corps of teachers “grouped together and suddenly had this camaraderie, which is something that is part of our culture and that makes us successful,” Magaña said.

However, he said some striking teachers “were guilting teachers into joining for solidarity.” He added that teachers who crossed picket lines told him they felt alienated.

Linsey Cobb, a special education teacher and special education team leader at Kepner Beacon, disputes that. She said every teacher wholeheartedly supported each teacher’s decision.

Cobb herself was torn about striking. She said she stood with teachers fighting for a system they believed would pay them a better, fairer wage. But the third-year teacher ended up reporting to work as usual Monday morning, feeling too strong of a pull to fulfill her responsibilities supporting students with individualized education plans — the complex and sometimes confounding binding documents for students with special needs.

Cobb said she was not fully prepared by what she experienced that morning.

“Even though I am very close with my students, I felt incredibly isolated,” she said. “I got the weirdest feeling. I got a lot of, ‘Miss, why aren’t you striking? Don’t you believe what teachers are fighting for?’ I was like, ‘I do!’ I had a little bit of an internal struggle.”

After attending the big teachers union rally Monday at the Capitol, Cobb said she woke up Tuesday and decided to join her colleagues picketing, which she did for the strike’s duration.

The strike brought to the forefront just how different the two Beacon campuses are. At Grant Beacon, 80 percent of students qualify for subsidized lunch — slightly above the district average. That part of the city, like much of Denver, is gentrifying. The southwest Denver neighborhood around Kepner is not. The school is a safe harbor from violence and trauma.

About half of Grant Beacon students showed up for school during the strike, and six in 10 teachers joined the strike. Four miles and a world away at Kepner Beacon, 90 percent of students showed up for school — and all but a few teachers were out on strike.

Against the backdrop of the strike, Magaña said he emphasized that words matter. Everyone in the buildings, he thought, not just teachers, ought to be considered educators. That was the role everyone was thrust into — administrators, deans, and district central office staff who through no choice of their own had to cover for absent teachers. Magaña, too. He taught math.

“We maintained a positive culture through a really weird and complicated time,” said Tristan Connett, who as Kepner’s dean of students was pressed into service to teach eighth-grade reading and language arts. “Not just for students, but all the adults, everyone included.”

Outside Kepner Beacon each morning of the strike, teachers huddled over donuts and coffee. Parents brought them hand-warmers in the 20-degree chill. One teacher sat in her car with the engine running recording a video message to her students, telling them where she was and spelling out the day’s lesson plan before she joined the picket line, Cobb said.

The Beacon schools promote character-building and use personalized learning, using data and technology to tailor instruction to individual students. As “innovation schools,” the schools are exempt from some state laws and aspects of the teachers union contract. Both schools were “green,” the second-highest ranking, in the district’s most recent school ratings.

Cracks in school culture did show during the strike. Magaña said one teacher at Grant Beacon was hurt by the negative reaction he received from striking colleagues.

The strike’s sudden end just after 6 a.m. Thursday led to mixed messages and confusion about what was expected of teachers that day, deepening rifts at the Beacon schools.

Cobb, the Kepner special education teacher, said teachers somehow got what turned out to be incorrect information from the union that they couldn’t be late for the start of school if they wanted to return.

Many striking teachers did not come back to school Thursday. That was out of step with the district as a whole, which saw the vast majority of teachers back in classrooms.

Some Beacon teachers, Magaña said, “said they were mentally and physically exhausted.” What, he asked, does that tell everyone who took on unfamiliar roles to keep the schools open?

When teachers, administrators, and staff arrived for Friday morning’s meeting, they congregated at tables with colored pencils and “reflection forms.” Everyone was asked to write down answers to two questions: What did you learn about yourself? What did you learn about your colleagues?

“I also brought out the obvious — the elephant in the room,” Magaña said. “There are hurt feelings. There is resentment from teachers to staff to students to parents. That is something we can’t pretend isn’t there, and we put it out there and acknowledge it to move forward.”

The message from the network administration left a number of teachers disappointed.

“Every teacher who went out on strike believed in it, we got this victory, and it wasn’t celebrated as a whole,” Cobb said. “It was more like there was an acknowledgement of what we want to repair. OK, but we felt like we deserved a little celebration for what we accomplished.”

Several teachers took up administrators’ offers to speak in private throughout the day, and when everyone gathered to wrap things up, Cobb said there was acknowledgement of what teachers had accomplished. Magaña said Saturday he doesn’t regret starting off the day like he did.

“We had to acknowledge all of the feelings of the group,” he said. “It was about all of us working together for a common ground.”

Under the tentative deal union members are expected to vote on next week, all of the teachers in the Beacon network will see their base pay increase. The incentives Kepner Beacon teachers receive for teaching in a “highest priority” school will be slightly smaller but will continue.

After the Presidents’ Day holiday Monday, teachers and students will return to school on Tuesday and try to the maintain culture that has contributed to promising academic progress.

“It’s about trust,” Magaña said. “Some of it was cracked a little it. There was no contention in the room (Friday). It was really coming in with openness and willingness by everyone to say, ‘It’s done, and we did the right thing for ourselves. Now it’s time to come closer together.’”

“Normalcy will happen,” added Cobb, the teacher. “But it might take a bit.”

bonus

Aurora school district numbers shows some positive results from hard-to-staff bonus

Students work on algebra problems in a college-level course at Hinkley High School in Aurora.

When the Aurora school district offered some teachers and service providers a bonus for accepting or returning to hard-to-staff positions, the district saw less turnover in those jobs and had more of them filled by the start of the school year.

But the results weren’t consistent across schools, and there were differences in how teachers and other support staff responded to the bonus. Some schools still saw big increases in turnover. And the district still couldn’t fill all positions by the start of the school year.

In a report that district staff will present to the Aurora school board Tuesday, survey responses show the bonus was most influential for new special service providers, such as nurses, occupational therapists, or speech language pathologists. But only 33 percent of new teachers coming into the district said the bonus made an impact on their decision.

Aurora administrators refused to talk about the findings ahead of the board meeting. When the district first announced the bonuses, Superintendent Rico Munn said he had hoped the pilot bonus system would help the district attract more candidates, fill more vacancies, and retain more employees. The union objected to the bonuses. The union and the district begin negotiations next month on how to spend $10 million that voters approved to raise teacher pay.

An arbitrator ruled that the district should have negotiated the terms of the bonuses with the union first, but the school board refused to uphold the finding. District officials had indicated that the results of the pilot incentives would play a role in what changes they propose going forward, and it’s not clear where the school board, a majority of whom were elected with union support, will come down.

On a state and national level, incentives for teachers are being questioned after Denver teachers went on strike, in part over a disagreement about how effective incentives can be and whether that money is better spent on base pay. Ultimately, the tentative agreement that ended the strike on Thursday maintained a number of bonuses, including $2,000 for educators in hard-to-staff positions.

In the Aurora pilot program, the district offered a bonus for special education, secondary math and secondary science teachers at 20 targeted schools. If staff in those positions committed to returning to their job for this year, they could get $3,000. If they returned, but did not give an early commitment, the bonus would be $2,500.

The same rules applied for other positions such as psychologists, nurses, occupational therapists, and speech pathologists, but those employees were eligible at all district schools. New employees in those positions could get $2,500.

To pay for the bonuses, the district had set aside $1.8 million from an unexpected increase in revenue due in part to rising property values. The district only ended up spending about $1.1 million.

Among 229 eligible teachers, 133 returned to their jobs, committing early, and another 29 returned without making an early commitment, meaning about 70 percent of teachers were retained and received the bonus.

Of the 20 schools at which teachers of math, science, and special education received incentives, turnover went down at 13 schools, up at another five, and stayed the same at two.

Among 184 staff members in the other hard-to-staff positions districtwide, 141 returned to their jobs, or 77 percent, all of them committing early and receiving the higher bonus.

The report doesn’t compare those numbers with previous years’.

Ramie Randles, a math teacher, was at Aurora West Collegiate Prep last year and received the bonus. But, she says, she had already decided to return to the same job this school year even before she learned about the bonus.

“To be honest with you it’s nice to get a little extra, but it’s a very small amount that’s not going to sway me one way or another,” Randles said.

In the second quarter of the school year, she left her job at Aurora West and is now teaching math at North Middle School.

The bonus is offered at both schools, but it wasn’t a factor, she said.

“I just feel like I want to feel valued in a job,” Randles said. “If I’m feeling like I’m happy that affects not just me, it affects my students. It affects my coworkers.”

According to the district, 98.26 percent of those who received a bonus remain in the same position as of this week.

Fill rates, which represent how many of the district’s positions are filled by the start of the school year, show an increase, although often small, among all positions except for school psychologists.

Fill rates over time: Did Aurora have more positions filled at the start of this school year than in the past?

Position 16-17 17-18 18-19
Secondary math teachers at 20 schools 91.5% 92.6% 93.4%
Secondary science teachers at 20 schools 93.5% 93.8% 94.8%
Special education teachers at 20 schools 92.6% 89.4% 90.24%
Nurses, district-wide 87.3% 94.6% 98%
Occupational therapists, district-wide 95.4% 80% 96.1%
Psychologists, district-wide 94.4% 96% 95.4%
Speech language pathologists, district-wide 75% 81.4% 85.4%

Another goal of the pilot was to help the district save money by decreasing the use of contract agencies to fill important positions.

The report found that compared with last year, fewer positions were filled through contract agencies.

The Aurora district “was one of the few districts in the metro area that did not provide some form of differentiated pay or incentive for hard-to-fill subject areas,” according to the district. As examples, the report cites Cherry Creek, Denver, and Douglas school districts.

Bruce Wilcox, president of Aurora’s teachers union, said the union has “no interest in pay like Denver does.”

He is against the bonus because he disagrees with setting up different pay for people doing the same jobs in different schools, and because he doubts it will have a long- term effect.

“For some, maybe money was enough to lure them in, but will it be enough to lure them in over a period of time?” Wilcox asked. “Money’s nice and every teacher needs it, let’s be honest, but is it enough to make you continue to work if the leadership and culture aren’t there?”

Tuesday, Aurora staff will also present the school board with an update on overall strategies to improve teacher recruitment and retention. Among those strategies: the development of new training for principals, including on how to motivate and retain high-performing employees.

Another report on the pilot incentives will be prepared this fall with final numbers of how many teachers stayed.

Find turnover rates for the pilot, by school, in the district’s report below. Note: The colors in the second column represent a comparison over the prior year with green showing that it is a lower rate than in the past.