Behind the numbers

Why ‘personalized learning’ advocates like Mark Zuckerberg keep citing a 1984 study — and why it might not say much about schools today

PHOTO: TechCrunch/Creative Commons
Facebook founder Mark Zuckerberg.

Facebook founder Mark Zuckerberg made a bold statement in a recent essay: By giving students individual help, average students can be turned into exceptional ones.

“If a student is at the 50th percentile in their class and they receive effective one-on-one tutoring, they jump on average to the 98th percentile,” Zuckerberg wrote.

It’s a remarkable claim, one that strains the limits of belief. And for good reason: The results from the 1984 study underlying it have essentially never been seen in modern research on public schools.

Still, the results have become a popular talking point among those promoting the “personalized learning” approach that Zuckerberg’s philanthropy is advancing. One video created by the Chan Zuckerberg Initiative features an illustration of a 50 on a graph zooming upward to hit 98. The New Schools Venture Fund, another influential education group that backs personalized learning, cites the same work by Benjamin Bloom.

But a close look at the study raises questions about its relevance to modern education debates and the ability of new buzzed-about programs to achieve remotely similar results.

“If you’re really going to make these huge investments and huge pushes [based on this study], you might want to be absolutely sure that the analysis of that research is solid,” said Ben Riley, head of the group Deans for Impact and a skeptic of personalized learning.

Jim Shelton, who heads CZI’s education work, said in an interview that the organization relies on a great deal of other research, but highlights Bloom to illustrate in the best case scenario for what schools might accomplish.

“It stands to reason that many kids that currently perform at levels that we consider average or even below average could be performing at levels that we would consider superlative,” he said.

Questions then and now about the meaning of Bloom’s work

The conclusions on the effects of tutoring from Bloom’s widely-cited paper are drawn from two studies conducted by University of Chicago graduate students.

One of those studies is available online, but reading the other requires some sleuthing. (We ended up paying for access through a service that compiles dissertations.)

In both studies, students were taught novel subject matter — probability or cartography — using different methods over the course of a few weeks. Some students were taught in a traditional lecture style, others received “mastery-based” teaching, and others received small group tutoring.

On a final test, students who were tutored one-on-one or in small groups came out far ahead, and in some cases the average tutored student beat 98 percent of those taught in the traditional way. Students who received the mastery-based teaching — which overlaps with modern conceptions of personalized learning — also did much better, though not as well as those tutored.

Jim Shelton of the Chan Zuckerberg Initiative in one of the organization’s video, saying that the average student will move to 98th percentile with one-on-one tutoring.

The applicability of these studies today is an open question. Combined, the studies focus on just three schools and a few hundred students. And since this was done more than 30 years ago, things like what traditional instruction looks like may have substantially changed.

The papers include little information about those final tests, but it appears they were designed by the researchers, unlike a traditional standardized test. Researcher-created assessments on subjects that are totally new to students — like cartography and probability, in this case — tend to see students make the largest gains.

Bloom’s work also doesn’t focus on technology-based tutoring, a point personalized learning advocates usually acknowledge. “If it supports anything, it supports one-on-one human tutoring,” Riley said.

But what earned the most attention, then and now, is how big of an impact tutoring had on students. The difference between tutoring and traditional instruction after just three weeks was two standard deviations — to researchers, a truly incredible result. It means bringing students from average to exceptional.

“I’ve never seen a study in education that found effects in the range of two standard deviations, so it’s remarkable for that reason,” said Jon Guryan, a Northwestern professor who has done research on tutoring.

Another researcher, Robert Slavin of Johns Hopkins University, logged concerns about Bloom’s outsize claims as early as 1987. Focusing on such unusually large gains, he wrote, “is misleading out of context and potentially damaging to educational research,” since it could lead researchers to “belittle” more realistic results.

Guryan’s recent work, on tutoring of struggling students in Chicago, found what would normally be considered fairly large gains: about a quarter of a standard deviation on math standardized tests. Other recent research on intensive tutoring in public schools looks similar, in some cases showing even smaller effects. Meanwhile, studies on computer-based personalized learning have shown a range of effects — but none comes close to two standard deviations.

Bror Saxberg, CZI’s vice president of learning science, acknowledged that Bloom’s findings are bigger than in other research. But he said human and computer tutoring can have a substantial impact, pointing to a 2011 overview of research where results come close to a full standard deviation. (This overview included studies in a variety of contexts, including outside K-12 education.)

In sum, a number of studies suggest that Bloom’s huge results are not plausible to expect in public schools today, and they have rarely been seen in other research. Meanwhile, Zuckerberg, Shelton, and CZI’s public statements imply that, with the right tools, students could see similar off-the-charts improvements.

Can ‘personalized learning’ drive huge gains? Advocates hope so.

Shelton analogized Bloom’s work to the human quest to run a four-minute mile: a crazy-seeming goal that was eventually attained by a small number of elite runners.

“Everyone said it was impossible to break the four-minute mile, until somebody broke the four-minute mile,” Shelton said. “Someone has broken the four-minute and its equivalent and we need to figure out how to do it and how to get a lot more people to be able to do it.”

Many others also see Bloom’s research less as a precise accounting of the results of tutoring and more as a call to action. Indeed, most of Bloom’s paper amounts to him pondering a question philanthropists are grappling with today: How can schools get the benefits of individual tutoring without the prohibitive expense of actually hiring each student their own tutor?

“If the takeaway from Bloom is that by doing tutoring and mastery you’re going to get two [standard deviation] gains — I don’t think that’s the right takeaway,” said Todd Rose, a Harvard professor who has argued that schools need greater customization. (CZI has funded some of Rose’s work.)

The value of the study, he says, is that “it speaks to a very different view of human potential than is embedded in our current system.”

Debbie Veney, a spokesperson for New Schools Venture Fund, which is supported by CZI, had a similar take: “[Bloom’s results] inspired and challenged many to figure how to achieve similar conditions in a more cost-effective way — which spawned many creative concepts and efforts to scale similar results.”

That’s in line with CZI’s sweeping ambitions — “empower every teacher everywhere,” as described in one CZI video — and deep pockets.

Zuckerberg and his wife Priscilla Chan have pledged to donate 99 percent of their Facebook shares — worth an estimated $45 billion in late 2015 — to CZI over their lifetime. The organization — which also focuses on criminal justice, immigration, and economic policy — is expected to give “hundreds of millions of dollars” per year to education causes.

The group has already supported a number of tech-based approaches to school, including the Summit learning platform, a computer program created by a charter network to help teachers personalize learning. CZI has also tried to broaden the definition of personalized learning, funding organizations that offer free eye exams and small-group, in-person tutoring.

A spokesperson pointed to other research CZI relies on, including psychological studies from Rose and others on how children learn and develop and the work of Stanford professor Carol Dweck, which suggests that people with a “growth mindset” are more likely to succeed.

But Sarah Reckhow, who studies education philanthropy at Michigan State University, suggests that CZI’s ambitious goals will meet the hard realities of the classroom and fall far short of Bloom’s results.

“I do think they’re setting themselves up to fail,” she said. “If you look at educational research, if you look at what will most definitely vary once you to put something into practice … those effect sizes won’t be replicated, but also there will probably be some cases where it will not turn out well or there will be unintended consequences.”

Asked about his benchmarks for success, Shelton said it’s not clear yet what is possible.

“We’re at the beginning of our journey, not the end of our journey,” he said. “We are in the business of trying to figure out how to solve this problem that has never been solved before.”

Colorado Votes 2018

Amendment 73: Understanding the tax increase for education on your Colorado ballot

PHOTO: Erica Meltzer/Chalkbeat
Thousands of Colorado teachers protested for more education funding in April. What will voters say in November?

Colorado voters face an important education decision this November: whether to approve a major statewide tax increase for schools. This request represents the third time in recent years that Colorado voters have been asked to put more money into schools.

The last two times, they gave a resounding no. Amendment 73 comes on the heels of teacher protests here and around the nation that have raised awareness of low pay and other unmet classroom needs.

Proponents of the measure say Colorado schools can’t keep doing more with less and need new revenue to do right by students. Opponents say that raising taxes will hurt the state’s economic prosperity without necessarily improving student outcomes.

Here’s what you need to know to make a decision:

What does Amendment 73 do?

This measure would create a graduated income tax for people earning more than $150,000 a year and would raise the state corporate tax rate. It also would change the assessment rate — the portion of your property value that is taxed — for commercial and residential property.

Altogether, these changes are projected to raise an additional $1.6 billion a year for preschool through 12th-grade education. That’s in addition to the roughly $9.7 billion in federal, state, and local money that Colorado will spend this year on schools.

The amendment raises the base amount Colorado is required to spend on each student, and it also dedicates money to preschool spots, full-day kindergarten, students with disabilities, those learning English, and those identified as gifted and talented.

Why is this on the ballot?

Colorado’s Taxpayer’s Bill of Rights requires that all tax increases be approved by voters. As for this particular tax increase, Colorado funds its schools below the national average, and since the Great Recession, state lawmakers have diverted to other areas billions of dollars constitutionally due to education.

Proponents of the measure believe the only way to adequately fund Colorado schools is to tap into an additional revenue source, like these tax increases.

Opponents counter that administrative spending has grown faster than student population and teacher salaries, and that the state and school districts could free up money for classrooms by setting new priorities.

I see amendments and propositions on my ballot. What’s the difference?

Propositions become laws and can be changed by the legislature. Amendments become part of the state constitution and can only be changed by another vote of the people. Amendments need the approval of 55 percent of voters to pass, a higher bar than propositions that only require a simple majority.

How will the money be spent? What guarantees do we have that it will reach the classroom?

Amendment 73 requires that new money “supplement and not supplant” existing funding. That means the legislature cannot redirect current spending on education and replace it with this new funding source. The amendment says the legislature should adopt a new formula for distributing money to districts that takes into account student and district characteristics, but it doesn’t lay out exactly what that should look like.

In the meantime, Amendment 73 describes specific uses for $866 million in new revenue:

  • Base spending per student will go up from $6,769 to $7,300, a 7.8 percent increase
  • Funding for full-day kindergarten. Right now, districts get a little more than half a student’s worth of funding for each kindergarten student.
  • An 8.3 percent increase for preschoool, bringing the total to $131 million
  • A 6.8 percent increase for special education, bringing the total to $296.1 million
  • An 80 percent increase for gifted and talented programs, bringing the total to $22.5 million
  • A 93 percent increase for English language learners, bringing the total to $41.6 million

The extra money that districts currently receive for students with disabilities, those learning English and those identified as gifted accounts for a fraction of the additional cost of educating them, particularly in the case of students with more significant disabilities. Districts have to use tracking codes to account for this money and ensure it goes to its intended purpose. In some districts, additional money might translate into better services for these students, while others might use the additional dedicated funding to free up other money.

That leaves $738.6 million that can be spent on public education as determined by the legislature. Once that money lands in school district coffers, they have broad discretion over how to spend it. This is by design and part of an effort to get buy-in from around the state. Many school boards have passed non-binding resolutions promising to spend the money on teacher pay, more mental health supports for students, and lower class sizes.

In turn, opponents have criticized the lack of specificity as a blank check that won’t necessarily increase teacher salaries or improve student outcomes.

A recent analysis from EdChoice found that since 1992, teacher salaries in Colorado had fallen even as per-student funding and the number of administrators had increased. Colorado Department of Education records show that instructional staff — teachers, counselors, speech language pathologists, school nurses — increased by 14 percent between 2006 and 2016 while administrative staff increased by 34 percent. School administrators argue these positions are necessary to support the work that teachers do and keep districts in compliance with a host of new state and federal regulations. In smaller districts, administrators often wear multiple hats. When we ask teachers about this issue, some of them share the concern that too much money gets spent on central administration, even as they also believe schools need more money overall

You can look up how much your district spends here.

What does it mean when people say Colorado schools are ‘underfunded’? Compared to what? How underfunded?

There are several different ways to look at this. The National Education Association, the country’s largest teachers union, ranks Colorado 28th in per pupil spending when state, local, and federal money is combined and puts Colorado about $758 per student below the national average. Education Week does a more complex ranking that takes into account regional cost differences and puts Colorado nearly $2,800 below the national average. Colorado teacher salaries are among the least competitive in the nation, making it hard to recruit and retain educators. More than 100 of Colorado’s 178 school districts operate on four-day weeks.

Back in 2000, after previous years of budget cuts, Colorado voters passed a constitutional amendment that requires school funding to increase by population plus inflation. But starting with the Great Recession, Colorado lawmakers have not allocated all the money required by that amendment. Over the past 10 years, Colorado schools have missed out on $7.5 billion the law requires them to receive. The courts have upheld this budget maneuver. Money from Amendment 73 could not be reallocated during the next downturn, protecting schools but potentially creating other budget problems for the state.

Colorado also gets low marks on equity. Colorado spends much less money on education than most states with similar levels of wealth and economic activity. Per-student spending varies widely around the state, with rich districts often getting more state money than poor ones. Some districts have convinced voters to approve local property tax increases, while other have not — or have such low tax bases that voters would need to take on large increases to generate much benefit. The additional funding from these local tax increases varies from $32 to $5,024 per student.

Amendment 73 wouldn’t change these structural problems with school funding. It would give state lawmakers more money with which to level the playing field. Right now, sending more money to some districts would require reducing funding to others, creating a political minefield.

Will I pay more in income taxes if Amendment 73 passes?

People who earn up to $150,000 a year will keep paying the same 4.63 percent state income tax rate they do now. Those earning more will pay a sliding increase starting at 5 percent for income from $150,001 to $200,000 up to 8.25 percent for income over $500,000. Someone with taxable income of $200,000 would pay an extra $185 a year, while someone with $1 million in taxable income would pay an extra $24,395, according to a fiscal analysis by the state.

The increases will affect about 8 percent of individual and joint income tax filers. Amendment 73 does not include a provision to adjust the income threshold for inflation, so it’s possible that more taxpayers will pay these higher rates in the future.

This change would generate most of the new revenue under Amendment 73.

What’s the effect on corporate taxes?

Amendment 73 would raise the corporate income tax rate from 4.63 percent to 6 percent. You can see how that compares to other states’ corporate income tax rates here. The average corporate income taxpayer would owe an additional $14,139, according to state fiscal analysts.

Would Amendment 73 raise my property taxes?

This is a complicated question. Amendment 73 does not raise property tax rates anywhere in the state. But if it passes, residential property owners will pay more in 2019, while owners of non-residential property would pay less.

Amendment 73 fixes the assessment rate at 7 percent for residential and 24 percent for non-residential property. That’s lower than it is now, but other constitutional provisions would have pushed the residential rate even lower in 2019. By setting the residential assessment rate instead of allowing it to continue to fall, homeowners would pay more property taxes to schools than they otherwise would have, while non-residential property owners would pay less.

Exactly how much more or less will depend on your property value, real estate trends in your community, and local tax rates.

This represents a partial fix to a complicated fiscal problem that has bedeviled Colorado lawmakers and the administrators of rural taxing entities — school districts, fire protection districts, and others — for years.

In Colorado, your property is assessed at close to market value, but your local tax rate only applies to a portion of that value. That’s the assessment rate. Another constitutional provision known as the Gallagher Amendment ensures that non-residential property owners always pay a larger share of property taxes than homeowners. Since 1982, when the Gallagher Amendment was approved by voters, property values along Colorado’s developed Front Range have skyrocketed, putting the assessment ratios between residential and other property seriously out of whack. Those ratios apply statewide, and many rural communities have seen their already sparse tax base hollowed out.

In the case of schools, that’s meant the state government has had to backfill more and more money that used to be generated by local taxes. Amendment 73 includes a provision to hold the assessment rates steady just for schools for two reasons. One is that it provides property tax relief to ranchers and farmers, which the measure’s backers hope bolsters support in parts of the state that are traditionally more hostile to tax increases. The other is that it ensures the new tax revenue generated by the amendment doesn’t just backfill an ever-deepening hole in rural districts.

Residential assessment rates will continue to drop for other taxing entities, creating an even more complex system, unless the state succeeds in a more comprehensive Gallagher fix.

Don’t schools get a lot of marijuana money already?

The bulk of marijuana tax revenue for education goes to a program that helps schools pay for buildings and construction repairs. Districts apply and compete for grant money from the program, and in most cases have to put up some portion of the project’s cost.

Starting this year, 12.59 percent of marijuana tax revenue is also set aside for the regular education budget. That’s about $20 million a year at current rates. Marijuana money is also set aside for various grant programs including one that schools can use to help pay for health professionals such as counselors or nurses. As the state collects more marijuana revenue, the amounts set aside for the grant programs has increased.

However, the marijuana money available to schools represents a tiny fraction of total education spending, and most of it can’t be spent on basic needs like teacher salaries or classroom materials.

 

Why not Michigan?

As Michigan’s poorest 4-year-olds wait for classroom seats, free pre-K for all kids seems elusive

PHOTO: Christina Veiga
All New York City four year-olds — including these kids who attend school is in the city's education department headquarters — are guaranteed a spot in a city-funded pre-K. In Michigan, far fewer students have access to free preschool.

Michigan is the home to America’s most famous study on the benefits of early childhood education.

But when it comes to providing free prekindergarten for all children, other states and cities are leading the way.

Vermont, Florida, Washington, and the District of Columbia have public programs for all 4-year-olds, regardless of income. Seven more states have greatly expanded their pre-K programs, too, including Wisconsin, where free voluntary pre-K is in the state’s 1848 constitution.

But not Michigan. Not yet, at least.

The pioneering Perry Preschool Study began in Ypsilanti in 1962 and followed 123 study participants starting at age 3 through the age of 40. Among the study’s  findings: Those who went to pre-K were more likely to graduate from high school and less likely to repeat grades. They were also less likely to use drugs or commit crimes.

As they grew older, they were more likely to be employed and to have stable homes, savings accounts, significantly higher incomes, and report good relations with their families.

Skills such as cooperative play lay the groundwork for children to get along with others. In addition, learning to use fine motor skills and mastering shapes, colors, numbers, and the alphabet, contribute to future growth.

Further research has underscored the worth of pre-K, making it a rare realm of bipartisan support. In fact, funding for early childhood education has risen under the past three governors.

“I’ve been around long enough to see Democrats and Republicans in office, and early childhood education continues to be on the radar as a positive,” said Lena Montgomery, director of the Wayne County branch of the Great Start Readiness Program, a state funded initiative for 4-year-olds from low-income families.

But even though the governor’s own 21st Century Education Commission recommended that Michigan expand pre-K with $390 million in new investment, he chose instead to further study the impact of Great Start. In his most recent budget, he allocated $300,000 to do that research, and kept spending for Great Start flat at $245.6 million.

Momentum toward providing publicly funded pre-K, often called universal preschool, has been slowed by cost, teacher shortages, and family resistance, advocates say. They also note that there is no incentive for different institutions to pool their money to pay for a more comprehensive pre-K program in the state.  

Other states and cities have navigated similar challenges. But Michigan families face a patchwork of options. They may keep young children at home, pay for private childcare or pre-K, or, if they meet income or disability requirements, they can enroll them in Great Start or federally funded Head Start. Both are designed to support vulnerable children, including families with low-incomes.

But there aren’t enough seats, even for every child in need. Great Start’s Montgomery said she has 27 programs with qualified families on wait lists. It’s common, she said, for policymakers to say they support children. But some families are still falling into the gaps because more money is needed, she said.

About 133,000 Michigan children are not enrolled in any early childhood program.

Half of Wayne County’s 3- and 4-year-olds are enrolled in various pre-K programs, said Iheoma Iruka of Highscope, though she added that “we can’t vouch for the quality of these programs.”

The education plan of Gretchen Whitmer, the Democratic nominee for governor, advocates for a universal program that expands Great Start until all 4-year-olds are eligible, similar to what the 21st Century Education Commission recommended. It would be paid for, according to her campaign staff, with anticipated increases in the School Aid Fund, which is mostly made up of sales, income, and property taxes. It would also use tax revenue from, among other things, the marijuana ballot initiative that’s expected to pass in November. Tax hikes shouldn’t be necessary, her staff said.

Bill Schuette, the Republican nominee, has an education plan that emphasizes third-grade literacy over pre-K. It mentions need-based transportation scholarships for preschoolers, and he said in a recent interview that universal pre-K was an option that he’d consider.

Hope Starts Here, the $50 million initiative created by the Kellogg and Kresge foundations to improve Detroit’s early childhood systems, has a number of suggestions to pay for universal pre-K. Among them: a dedicated tax proposal, a local sales tax on alcohol, coordinated philanthropic and corporate giving, and leveraging all federal grant money.

States and cities around the nation have experimented with other strategies. Georgia tied pre-K funding to the state lottery. New York City’s new universal program for all 3- and 4-year-olds comes from a mix of city, state and federal funding. Oklahoma, a pioneer in the field, discovered that school districts with half-day kindergartens were receiving state money meant for full-day programs. Lawmakers reformed the state aid formula so that those resources went into pre-K. (The districts had been spending the extra money on sports.)

Others have expanded access by combining different sources of money. North Carolina integrated pre-K with its K-12 schools and contributed part of the Title 1 money that’s allocated to school districts. Chicago is moving toward universal pre-K with a mix of state and district budget increases, and block grants. Washington, D.C. blends Head Start and local funding into its education formula.

A pilot model for blended funding in Michigan can be found in Flint, where the state’s only Educare program is based on the grounds of a former elementary school. The national Educare Early Learning Network draws from multiple revenue sources, including federal, state, and philanthropic dollars.

But regardless of where the money is coming from, opportunities to expand pre-K programs may be missed because of the statewide teacher shortage. In addition,  salaries are not as high as they are in K-12 schools. The median salary for Head Start teachers is $27,613, and for lead Great Start teachers, $37,440, according to a statewide advocacy organization.

To recruit and retain more teachers at all levels, including pre-K,  a new public-private initiative called Teach 313 launched in Detroit in August. Other places facing shortages or high turnover for its preschool teachers have turned to Teach for America to fill gaps, or provided scholarships for early childhood educators to obtain degrees that would raise their wages.

But before Michigan can explore other strategies and expand into universal pre-K, it needs to make the program it already has available to more families.

If you ask Montgomery from Great Start about her wish list, it begins with providing pre-K to all the children who are sitting on waitlists.

“It would be wonderful to to say to parents, ‘We have a spot for your child,’” Montgomery said. “ ‘You don’t have to wait for someone to drop out or leave.’ It would be wonderful to say to the people who want to run programs, or to expand their programs in their communities, ‘We have the funds for you set up and run a high quality program.’ ”