A coalition of rural Colorado districts has spent four years building a parallel, unofficial school improvement system that looks beyond test scores and seeks to measure things like the quality of instruction and student attitudes toward learning.
“We want data that helps us improve, and state data is not helping us do that,” said Lisa Yates, superintendent in the 900-student Buena Vista school district, a founding member of what’s known as the Student Centered Accountability Project.
A bill making its way through the Colorado legislature with bipartisan support would lend financial and logistical support to these efforts and encourage other districts and charter schools to undertake their own experiments. Senate Bill 204, sponsored by state Sen. Tammy Story, a Conifer Democrat, would create a $500,000 grant program that school districts, educational cooperatives, and charter schools could tap to cover the costs of developing alternative accountability systems.
The bill also calls for annual reporting and an annual meeting of the participating districts and schools to learn from each other and recommend potential changes in state rules or laws related to accountability. In the third year of the program, the state would hire an outside evaluator to review the experiments’ impact on student success and on schools’ ability to improve continuously.
These pilot programs would supplement, not replace regular state ratings, which are primarily based on student performance on standardized tests. But supporters hope the experiments will eventually lead to changes in the statewide system.
“It’s part of our culture as a state to innovate locally and then let the state as a whole benefit from that,” said Julie Oxenford O’Brian, director of the Center for Transforming Learning and Teaching at the University of Colorado Denver, pointing to the way Colorado measures student growth as one example. “It’s happening now, but what we haven’t had is a way for others to learn from what they’re doing.”
Unlike another effort to move away from test-based accountability, changes to the teacher evaluation system that were ultimately unsuccessful, the accountability pilot has garnered broad support by leaving the status quo intact while promoting local experimentation.
“We believe the systems by which we hold school accountable should include multiple measures and reflect the whole child,” Leslie Colwell of the Colorado Children’s Campaign told the Senate Education Committee earlier this month.
The participants in the Student Centered Accountability Project started working together because as small, rural districts, they found state tests more burdensome and less useful. One or two students who score particularly high or low can have a big impact on aggregate test scores when a district only has a few hundred students. The way the state compensates for that, with three-year averages, makes it hard to track changes from year to year.
The districts worked with education researchers to develop other criteria and ways to measure them consistently over time. Yates said educators thought about the qualities they would want to see in their own children, like grit, perseverance, and the ability to relate well to their peers, and then thought about how to set up school environments that promoted those qualities.
Teams from participating districts, along with state education officials, visit each other’s schools, observe classrooms, and provide feedback. Schools also conduct teacher and student surveys.
Yates said one district that scored low on the quality of instruction and curriculum started working with a consultant to improve curriculum and added training days for teachers. The district made those changes within a matter of months, less time than it takes for the state to return the results of standardized tests.
“These peer-to-peer evaluations hold your feet to the fire,” said Rob Sanders, superintendent of the 300-student Merino district in northeast Colorado. “It’s not the fear of a poor rating and the state coming in. It’s the fear of looking bad in front of your peers. If you say you’re going to do something, by golly, when they come back, you better have done it.”
Oxenford O’Brian, who has worked on the Student Centered Accountability Project, said this approach promotes meaningful change in schools and addresses what she sees as a problem with the current accountability system.
“Whether it’s intended or not, our current system incentivizes districts and schools that are trying to get out of the danger zone to do things to gain points instead of what might be the most important thing to improve,” she said.
It’s not clear yet how these experiments might change the state accountability system. There are no immediate plans to revamp it, and the last time members of the State Board of Education asked tough questions about school ratings, they pushed for more reliance on test scores, not less.
Lisa Medler, executive director of improvement planning in the Colorado Department of Education, said officials there want to learn from local experiments, but they also want to maintain apples-to-apples comparisons across districts and schools.
“The department’s stance is currently that our accountability system is based on having some consistent measures across all schools and districts,” she said. “The emphasis needs to be on those consistent measures so we have a single accountability system.”
Oxenford O’Brian said there needs to be “flex” in the system.
“It’s pretty easy to devolve to the easiest things to measure with technical rigor and just use those,” she said. “That’s the trap that we’re in. We have to ask: Are we putting more value in comparable ratings than in data that supports improvement? Are we doing the right things to help improvement happen, or are we placing higher value on measures we can compare?”