Report: what other states can learn from NYC's data systems

Acknowledging that asking teachers to analyze student data has not fundamentally changed how they teach, Department of Education officials are beginning to change their approach. And other states just beginning to build their own student databases can learn from the city’s pivot, according to a report out today from Education Sector, a D.C.-based think tank.

The report recounts the brief history of New York City’s adventures with its own database known as ARIS, for Achievement Reporting and Innovation System. It also looks at schools’ gradual adoption of “data inquiry teams,” which about 65 percent of teachers were using at the end of last year.

Inquiry teams are groups of four or five teachers that select a small number of low-performing students to focus on. With information culled from ARIS, the teachers try and alter curriculum and teaching methods to improve the students’ performance. These teams are the DOE’s largest-scale reform that directly targets the instructional process. Officials hope to bring the participation rate up to 90 percent by the end of this year.

One DOE official notes in the report, “data analysis ‘is not yet leading to fundamental change in teacher practice or decision-making.” The reports states that some of the reason for that is that ARIS isn’t showing teachers as much data as they want to see as fast as they want to see it. But the city clearly expects that to change as it opens ARIS up to data that comes from teachers, not just to them. Here are some of the changes the city has in store:

  • Responding to complaints that ARIS’s data isn’t updated frequently enough and is too broad to really help inquiry teams, the city is creating ARIS Local, which we reported on last month. Though several years away, ARIS Local will eventually let teachers load their own data onto the city’s servers. This could eventually give teachers the ability compare how a student does on a test the teacher made herself to how he does on the state’s exams.
  • One drawback of ARIS that the city is beginning to look at is the fact that only teachers, administrators, and parents can access the data. Community and after-school organizations that work with students can’t see their attendance information, or whether they’re classified as English language learners. But that is changing, as Tucker writes:
  • “Sophie Lippincott, former director of knowledge sharing in the Division of Performance and Accountability, sees the clear value of sharing ARIS information with community-based organizations, and she has been trying to begin a program to do so. “It’s obviously in our favor to have partner organizations using ARIS,” she says. The district has trained two organizations that are “gung-ho . . . and ready to go,” she says. But, here again, organizational silos are proving difficult to break. The developers of ARIS did not contemplate out-of-school use; because user authentication is based on the Education Department’s human resources databases, it is difficult for non-school employees to gain access. (The district has recently developed a temporary solution that enables schools to grant access to certain community partners.)”
  • Though the DOE hasn’t have the tracking programs it needs to know what parts of the site are getting the most use or how parents are viewing ARIS, officials do know that more parents are logging in. The report states:
  • “A total of 62,000 unique users logged in to the ARIS educator tools from July 2009 to March 2010. As of August 2009, 340,000 different parent accounts had been accessed at least once, most often during parent/ teacher conferences.”
  • The report gives examples of some ways the city has tried to get low-income families to check out their students’ scores.
  • “Parent Link employed more strategies in a pilot effort to boost use by low-income families in 24 schools. Successful strategies include using parent and student volunteers; in one school, students train their parents and have them sign notes confirming they have logged in to Parent Link. Teachers have been trained in how to talk about data with parents. Another school opened its library early for ARIS workshops, and at another, a parent coordinator e-mailed parents who had not logged in. One school, located across from a homeless shelter, even created a resource room with a washer, dryer, and Internet access.”