From the Statehouse

Ready or not, online tests coming to Colorado

Colorado kids will get a chance to put their computer skills – and their academic knowledge – to high-stakes use when the state’s first online tests roll out later this school year for some students.

The plan is for the tests to be piloted this year followed by statewide testing in 2015.

Testing illustrationSo by the spring of 2015 more Colorado students will be taking more tests than under the current TCAP system, and they will be taking those tests on computers and tablets.

And, those students likely won’t be scoring as well as they did on the TCAPs, given that the new tests, partly based on the Common Core Standards, will be testing different things and are supposed to be harder than exams given in the past.

In the meantime, administrators and tech staff in districts across the state are fretting as they try to gear up for a whole new testing environment. Districts are worried about whether they have enough bandwidth and enough computers, but some administrators are even more concerned about the planning and tight scheduling that will be needed to fit online tests into school calendars.

“The general state of readiness is – we’re probably at a C-minus,” said Steven Clagg, chief information officer for Aurora Public Schools and president of the Colorado Association of Leaders in Education Technology. Clagg tempers his remark by noting his impression is based on anecdotal evidence, adding, “I think the state of Colorado is going to make it through this, but it won’t be without pain.”

The first wave of about 240,000 students will face new tests in April 2014 when students in four grades will take social studies or science tests. High school seniors will take both of those tests in the fall of 2014.

And, in the spring of 2015, all Colorado students in grades three through 11 will take online tests in language arts and math. Those will replace the TCAP reading, writing and math tests currently given to students in third through 10th grade. The new tests are being developed by the Partnership for Assessment of Readiness for College and Careers (PARCC).

The social studies and science tests are launching during the same school year that districts are implementing new content standards on which the PARCC tests will be based, rolling out upgraded early literacy programs for students in kindergarten through third grade and evaluating all principals, assistant principals and teachers using systems based on the landmark 2010 educator effectiveness law. All of those mandates have ratcheted up the level of anxiety in school districts statewide. (See this EdNews story about the new evaluation systems.)

Here’s a look at the some of the features of and the issues raised by the new tests.

Testing the tests

The science and social studies exams were field tested last April in 261 schools, where 25,408 sample tests were given.

“We were very pleased with how that spring field test went,” said Joyce Zurkowksi, Colorado Department of Education executive director of assessment. “Our small districts did fine.” Clagg agrees that the spring tests provided valuable lessons.

The Hanover district southeast of Colorado Springs participated in the field test. Paul McCarty, superintendent of the 227-student, two-school district, said the experience highlighted the challenges of online testing.

McCarty’s district doesn’t have a technology staffer and has to rely on help from the area board of cooperative educational services. He’s worried about smaller districts like his having to rely on staff members who don’t have the appropriate tech background.

More sample tests will given this autumn, at elementary and middle schools during the second half of September and in high schools starting in late October.

Short sample tests also are available online (check them out here), and CDE is encouraging schools to have students practice on those tests so they can become familiar with the look and feel of the exams.

Who’s tested and when

The TCAP reading, writing and math tests, like the CSAPs before them, have been given every spring to students in the third through 10th grades. Students have taken science tests in grades five, eight and 10. Last spring about 500,000 students took a total of about 1.7 million exams.

That schedule is shifting.

The social studies tests will be given in the fourth and seventh grades. While science will continue to be tested in fifth and eighth grades, 10th graders will be off the hook. Instead, social studies and science tests will be given to high school students in the fall of senior year, the first time that statewide tests have been given to 12th graders.

Next spring will mark the first statewide testing on social studies; science tests have been around awhile but previously have been given on paper.

The new PARCC tests will be expanded to include 11th graders, who also will continue to take the ACT college-readiness test.

What the tests will be like

The biggest change for students, of course, will be taking tests on computers rather than with pencil and paper.

Tests will include multiple choice items, some of which will require students to move suggested answers around on the screen, as well as questions that involve reading a text and writing an answer.

The timing of PARCC tests also will be different from the TCAPs. The new exams will be given in two segments, one about three-quarters of the way through the school year and the second closer to the end of the year. Districts will be allowed “testing windows” of a few weeks intended to give them enough time to test students with the equipment available.

Students also are likely to spend a bit more time on the PARCC tests than they do on TCAPs. (See this CDE chart for a comparison.)

It’s estimated that total PARCC testing time will be eight hours for third graders, just over nine hours in grades four and five, a little less than nine and a half hours in middle school and just over nine and a half hours in high school.

Clagg and others believe the new tests will require a whole new approach by teachers and administrators.

For instance, he said, “Proctoring is going to be a lot different. … We believe you’re going to have to have two people in a room or maybe more.”

He also noted that computers often are packed closely in school computer labs. “You’re going to have kids shoulder to shoulder taking this tests.” He said that during testing this spring students sometimes looked at neighboring screens. “They weren’t trying to cheat. They just got distracted for a minute” by movement or a flash of color on a neighboring monitor.

Sample test question
Picture of sample question from new Colorado science tests.

So, Clagg said, proctors will have to learn a new set of skills, including restarting computers when they lock up, to effectively monitor online testing.

He also said the school tech staffers – or administrators in districts without the luxury of IT experts – will have to learn how to prepare computers ahead of time. Automated processes like an anti-virus scan will have to be turned off, he said, because they will lock a test if they fire up while a student is working.

Clegg also said schools will have to get highly organized about test scheduling in order to make the most efficient use of the computers available.

Hanover’s McCarty is worried more about scheduling than he is about tech issues like bandwidth.

“I think the scheduling of classes is huge,” he said. “We don’t have that many computers, and we try to use and teach and use technology as much as we can.” He noted that reserving a computer lab for testing will make it unavailable for instructional use.

Clagg echoes that concern. “What about the classes that were going to use that lab? A lot of the challenge is going to be, what’s the ripple effect of the test? … We’re going to have to be really creative in the districts.”

Will school districts be ready?

Zurkowski believes school districts will be ready for next spring’s social studies and science tests, which she said “will help us gauge what’s needed for PARCC. …We are really expecting that by 2015 we will have worked through some of the hiccups.”

Based on data compiled from a sample of districts, CDE estimates 43.1 percent of middle and elementary schools can complete the science and social studies tests in less than five days, and 51.4 percent can do that in six to 10 days.

The department also estimates 85.5 percent of high schools can do 12th grade testing in under five days and that 10.5 percent can do so in six to 10.

Next spring’s tests are being provided by Pearson Education, one of the nation’s largest testing companies. Zurkowski said Pearson also is expected to provide the PARCC tests, so similarity in test format and technical specifications should be an advantage for Colorado.

There’s been a lot of concern about districts’ access to high-speed Internet connections – bandwidth – but Zurkowski believes one Pearson technology should ease some of those concerns.

The company offers what’s called “proctor caching,” which means districts can download full sets of test materials and that students can take the exams on a local network and not have to rely on a fast Internet connection. Once testing is finished the completed tests are transmitted back to Pearson.

“The proctor caching is going to help us a lot in this state,” she said. For instance, CDE estimates that in a sample district only 76 tests could be taken over the Internet simultaneously but that more than 1,000 machines could access a test on a local district network.

Clagg agrees that caching will help districts but notes, “It takes a bit of knowledge to get it implemented.”

Beyond bandwidth, Clagg notes that schools face other challenges. “Do districts have enough devices?”

CDE has calculated generous testing windows to account for limits on the number of computers that districts have. Clagg thinks those windows are good, but “the key thing is going to be scheduling.”

Asked if districts plan to buy new equipment for testing, Clagg said, “Everybody’s thinking about that.” Testing “is making us ask harder questions about the devices we’re buying. We have to look at multipurpose devices.”

McCarty said he thinks Hanover is ready for spring 2014 – if everything goes right – but that the PARCC tests in 2015 present a different challenge.

“At some point we’re going to have to set aside more money for computers and convert another classroom to a computer lab.”

The cost of new tests

PARCC announced last month that the estimated cost of its English and math tests would be $29.50 per student, just below the current median cost of $29.95 for states participating in the multi-state group.

Zurkowski said that’s “pretty comparable to what Colorado is used to,” citing a figure of $29.25.

The department’s 2014-15 budget request, which will be considered by the 2014 legislature, is expected to include $3.4 million in additional funds for the PARCC tests. Zurkowski said that covers addition of 11th grade tests and a bit extra to pay for paper tests for districts that may not be ready for online. (The PARCC paper tests come to about $34 per student.)

CDE also is likely to request an additional $300,000 for increased costs of ACT and English language proficiency tests, plus $3 million for early literacy assessments.

The state currently budgets about $34 million a year for all testing costs, including the TCAPs, science and social studies, work on “augmentations” to the PARCC tests, development of Spanish literacy assessments, alternate assessments for students with significant cognitive disabilities, ACT tests and ACCESS, an English language proficiency assessment. (The federal government provides about $7.6 million of the testing budget.)

Districts long have complained that they also bear the burden of testing costs, but it’s hard to put numbers on that.

Teachers and other professionals, of course, are on the payroll whether they’re proctoring tests or giving normal classroom lessons.

But there are other costs as well. A cost study done for the plaintiffs in the Lobato v. State school funding lawsuit estimated that schools, depending on size, bear non-personnel assessment costs of between $25 and $50 a student. Zurkowski said CDE hasn’t researched district testing costs.

Asked about district costs for the coming new tests, Clagg said, “I have not come up with a forecasted cost for this [but] there is going to be a cost for districts,” including for training.

The bigger issue, Clagg said, is that “The cost is an opportunity cost. What aren’t we doing so we can do this?”

New tests equal lower achievement

One of the final slides in a CDE presentation at a recent Colorado Association of School Executives convention gives this gives this understated warning:

“Given the increased rigor of both the Colorado Academic Standards and the new assessments, it is reasonable to prepare for a drop in performance based on them compared to the performance demonstrated under the current system.”

Department officials have been sounding that warning for months in a variety of meetings, and it’s commonly accepted that average performance drops when tests are changed, particularly when new exams test new skills.

Proficiency levels dropped in New York State after students took new tests based on the Common Core Standards. (See this Gotham Schools story for details.) Kentucky had a similar experience last year after students took new tests.

Tracking and using the data

While achievement levels are expected to drop, results on old and new tests aren’t necessarily comparable because they’re based on different standards, CDE officials say.

Past TCAP results “are going to become irrelevant in a very short period of time,” Zurkowski said. “Can we compare those numbers?” – referring to percentages of students scoring proficient or advanced. “Probably not.”

CDE officials are more concerned about the Colorado Growth Model, which calculates student test score growth over multiple years to determine if students are making sufficient progress. Growth is a key factor in the state’s accountability and rating system of districts and schools.

Alyssa Pearson, director of CDE’s accountability and data analysis unit, said the department is working on how to make the growth model bridge TCAP results and the new tests. “You can do it,” she said. Pearson noted that Colorado recently switched English language proficiency tests and was able to track most growth data across the two sets of data.

But, she said, calculating one key piece of growth data – whether a student is making adequate growth toward proficiency – probably will require two years of data from the new tests.

What if new tests aren’t ready?

The PARCC tests – and the Common Core Standards on which they’re based – have come under fire recently in some states. (See this EdNews story.)

Robert Hammond
Robert Hammond / File photo

During a recent State Board of Education budget session, chair Paul Lundeen asked if CDE should set money aside to pay for other tests if the PARCC exams aren’t ready in 2015.

Education Commissioner Robert Hammond, who sits on PARCC’s board, said, “So far my belief is they will make it.”

If that doesn’t happen, Colorado has other options, Associate Commissioner Jill Hawley told the board. “We may some options; we could still procure tests with another vendor. … The potential for getting something off the shelf exists.”

good news bad news

Most Tennessee districts are showing academic growth, but districts with the farthest to go improved the least

PHOTO: Alan Petersime

It’s not just Memphis: Across Tennessee, districts with many struggling schools posted lower-than-expected growth scores on this year’s state exams, according to data released Tuesday.

The majority of Tennessee’s 147 districts did post scores that suggest students are making or exceeding expected progress, with over a third earning the top growth score.

But most students in three of the state’s four largest districts — in Memphis, Nashville and Chattanooga — aren’t growing academically as they should, and neither are those in most of their “priority schools” in the state’s bottom 5 percent.

The divide prompted Education Commissioner Candice McQueen to send a “good news, bad news” email to superintendents.

“These results point to the ability for all students to grow,” she wrote of the top-performing districts, many of which have a wide range of academic achievement and student demographics.

Of those in the bottom, she said the state would analyze the latest data to determine “critical next steps,” especially for priority schools, which also are located in high-poverty communities.

“My message to the leaders of Priority schools … is that this level of growth will never get kids back on track, so we have to double-down on what works – strong instruction and engagement, every day, with no excuses,” McQueen said.

Growth scores are supposed to take poverty into account, so the divide suggests that either the algorithm didn’t work as it’s supposed to or, in fact, little has happened to change conditions at the state’s lowest-performing schools, despite years of aggressive efforts in many places.

The results are bittersweet for Tennessee, which has pioneered growth measures for student learning and judging the effectiveness of its teachers and schools under its Tennessee Value-Added Assessment System, known as TVAAS.

On the one hand, the latest TVAAS data shows mostly stable growth through the transition to TNReady, the state’s new test aligned to new academic standards, in the first year of full testing for grades 3-11. On the other hand, Tennessee has invested tens of millions of dollars and years of reforms toward improving struggling schools — all part of its massive overhaul of K-12 education fueled by its 2009 federal Race to the Top award.

The state-run Achievement School District, which launched in the Race to the Top era to turn around the lowest-performing schools, saw a few bright spots, but almost two-thirds of schools in its charter-reliant portfolio scored in the bottom levels of student growth.

Shelby County’s own turnaround program, the Innovation Zone, fared poorly too, with a large percentage of its Memphis schools scoring 1 on a scale of 1 to 5, after years of scoring 4s and 5s.


District profile: Most Memphis schools score low on student growth


Superintendent Dorsey Hopson called the results a “wakeup call” for the state’s biggest district in Memphis.

“When you have a population of kids in high poverty that were already lagging behind on the old, much easier test, it’s not surprising that we’ve got a lot of work to do here,” he said, citing the need to support teachers in mastering the state’s new standards.

“The good part is that we’ve seen the test now and we know what’s expected. The bad part is we’ve seen the test … and it’s a different monster,” he told Chalkbeat.

You can find district composite scores below. (A TVAAS score of 3 represents average growth for a student in one school year.) For a school-by-school list, visit the state’s website.

exclusive

Most Memphis schools score low on student growth under new state test

PHOTO: Stephanie Snyder

More than half of Memphis schools received the lowest possible score for student growth on Tennessee’s new test last school year, according to data obtained by Chalkbeat for Shelby County Schools.

On a scale of 1 to 5, with 1 being the lowest measure, about 54 percent of the district’s 187 schools scored in the bottom rung of the Tennessee Value-Added Assessment System, known as TVAAS.

That includes most schools in the Innovation Zone, a reversal after years of showing high growth in the district’s prized turnaround program.

Charter schools fared poorly as well, as did schools that were deemed among the state’s fastest-improving in 2015.

Superintendent Dorsey Hopson called the scores a “huge wakeup call.”

“It shows that we’ve got a tremendous amount of work to do,” Hopson told Chalkbeat on Monday. “It’s going to be hard and it’s going to be frustrating. … It starts with making sure we’re supporting teachers around mastering the new standards.”

District leaders across Tennessee have been trying to wrap their heads around the latest growth scores since receiving the data in late August from the State Department of Education. Only two years earlier, the Memphis district garnered the highest possible overall growth score. But since then, the state has switched to a harder test called TNReady that is aligned for the first time to more rigorous academic standards.

TVAAS results are scheduled to be released publicly this week, but Chalkbeat obtained a copy being circulated within Shelby County Schools, Tennessee’s largest district.

The data is prompting questions from some Memphis educators — and assurances from state officials — over the validity of TVAAS, the state’s system for measuring learning and judging the effectiveness of its teachers and schools.

This is the first year of issuing district-wide TVAAS scores since 2015. That’s because of the state’s cancellation of 2016 testing for grades 3-8 due mostly to failures in the switch to online testing.

Some educators wonder whether the bumpy switch to TNReady is a factor in this year’s nosedive, along with changes in how the scores are calculated.

For example, data for fourth-graders is missing since there is no prior state testing in third grade for comparison. Elementary and middle schools also don’t have growth scores for social studies, since the 2017 questions were a trial run and the results don’t count toward a school’s score.

Hopson acknowledged concerns over how the state compares results from “two very different tests which clearly are apples and oranges,” but he added that the district won’t use that as an excuse.

“Notwithstanding those questions, it’s the system upon which we’re evaluated on and judged,” he said.

State officials stand by TVAAS. They say drops in proficiency rates resulting from a harder test have no impact on the ability of teachers, schools and districts to earn strong TVAAS scores, since all students are experiencing the same change.

“Because TVAAS always looks at relative growth from year to year, not absolute test scores, it can be stable through transitions,” said Sara Gast, a spokeswoman for the State Department of Education.

Shelby County Schools is not the only district with disappointing TVAAS results. In Chattanooga, Hamilton County Schools logged low growth scores. But Gast said that more districts earned average or high growth scores of 3, 4 or 5 last school year than happened in 2015.

Want to help us understand this issue? Send your observations to [email protected]

Below is a breakdown of Shelby County’s TVAAS scores. A link to a school-by-school list of scores is at the bottom of this story.

Districtwide

School-wide scores are a combination of growth in each tested subject: literacy, math, science and social studies.

Fifty three schools saw high growth in literacy, an area where Shelby County Schools has doubled down, especially in early grades. And 51 schools saw high growth in math.

Note: A TVAAS score of 3 represents average growth for a student in one school year. A score of 1 represents significantly lower academic growth compared to peers across the state.

2017

School-wide composite Number of schools Percent of schools
1 101 54%
2 19 10%
3 20 11%
4 10 5%
5 37 20%

2015

School-wide composite Number of schools Percent of schools
1 58 28%
2 16 8%
3 38 19%
4 18 9%
5 75 37%

Innovation Zone

Out of the 23 schools in the district’s program to turn around low-performing schools, most received a growth score of 1 in 2017. That stands in stark contrast to prior years since the program opened in 2012, when most schools were on a fast growth track.

School-wide composite Number of iZone schools
1 14
2 2
3 2
4 0
5 5

Reward schools

Nearly half of 32 schools deemed 2015 Tennessee reward schools for high growth saw a major drop in TVAAS scores in 2017:

  • Central High
  • Cherokee Elementary
  • Germanshire Elementary
  • KIPP Memphis Middle Academy
  • Kirby High
  • Memphis Business Academy Elementary
  • Power Center Academy High
  • Power Center Academy Middle
  • Ross Elementary
  • Sheffield High
  • South Park Elementary
  • Southwind High
  • Treadwell Middle
  • Westside Elementary

Charter schools

Charter schools authorized by Shelby County Schools fared similarly to district-run schools in growth scores, with nearly half receiving a TVAAS of 1 compared to 26 percent of charter schools receiving the same score in 2015.

2017

School-wide composite Number of iZone schools
1 18
2 6
3 7
4 2
5 7

2015

School-wide composite Number of iZone schools
1 10
2 2
3 7
4 3
5 16

Optional schools

Half of the the district’s optional schools, which are special studies schools that require students to test into its programs, received a 1 on TVAAS. That’s compared to just 19 percent in 2015.

2017

School-wide composite Number of iZone schools
1 23
2 6
3 5
4 2
5 10

2015

School-wide composite Number of iZone schools
2 5
3 6
4 5
5 14

You can sort through a full list of TVAAS scores for Shelby County Schools here.