A few years ago, North Carolina school officials looked at a group of the state’s struggling schools and decided: we need to help.
State officials designed a turnaround strategy for those 75 schools, which were largely rural and served mostly low-income students of color. They dispatched education department staffers to analyze what was going wrong and make a plan, and then sent coaches to work with teachers and principals to improve instruction.
Their approach would seem innocuous at worst and genuinely helpful at best.
But the results look quite different. The program actually caused test scores to fall modestly and teacher turnover to rise sharply at those schools, according to new research.
To Gary Henry of Vanderbilt University, one of the researchers, the finding underscores that intervening in schools can backfire, and it’s not always obvious what schools need to improve.
“‘I always thought that it was better to do something than to do nothing,’” Henry said a North Carolina policymaker recently told him. “But it appears from this,” Henry said, “it actually sometimes could be worse to do something.”
The research adds to the current state of confusion about how best to improve struggling schools at a moment where the strategies states are using are in flux. Under the Obama administration’s federal turnaround program, states’ lowest-performing schools were required to fire the principal, and in some cases half the existing staff, and they received an infusion of extra resources. But now the federal education law, the Every Student Succeeds Act, is relatively light on requirements.
A number of states are set to use programs that look more like North Carolina’s effort, including ones that rely on coaching, raising questions about their likelihood of success.
“These findings may serve as a cautionary tale for how states support their lowest-performing schools under ESSA,” write Henry and Erica Harbatkin of Vanderbilt.
The pair of researchers tracked state test scores of the schools in the program and compared them to those at similar schools that weren’t part of the turnaround. By year two, test scores were moderately lower across subjects as a result. Teacher turnover, meanwhile, had increased by over 20 percentage points to around 40% — a number that included some of the schools’ top teachers.
In a separate paper, Henry and Harbatkin found that the initiative also didn’t improve results on an early-grade reading assessment.
What went wrong?
Andre Stewart is the principal at Weldon Middle School, a school in northeast North Carolina that was part of the program. He said that he found the coaching helpful, but he can understand why it might have created challenges elsewhere.
“It was a process of having the teachers buy in,” he said. “At times there were mixed reviews on things because the coach might come in and say, ‘I worked in another place and we did this way.’”
The state officials’ school improvement plans also invited mixed reactions, Stewart said, if they failed to take into account local context. The report might ding a school for a delayed transition between classes, for instance, but what the officials didn’t realize is that students were held for extra time to discuss a gang issue happening in the neighborhood.
“Some of the things in the report were spot on,” he said. “Some of those things in the report we would have to look at through the lens of, they’re coming from somewhere else and they may not understand our culture and our climate in our school and our community.”
Deanna Townsend-Smith, the North Carolina state board of education’s director of operations and policy, isn’t convinced the program didn’t work. “I would strongly disagree with the statement that the turnaround was seemingly unsuccessful as other factors should be considered,” she said.
She pointed to a presentation Henry gave to the state last year, though its content largely mirrors the study. One difference: The presentation notes that the turnaround schools did see their student proficiency rates increase — though to a smaller degree than similar schools that weren’t part of the program.
Henry says the North Carolina coaches were spread much more thinly than in the state’s previous turnaround initiative, which was funded by the federal government and required the use of one of the more disruptive turnaround models. (This earlier effort had 150 coaches working across 107 schools, compared to 32 coaches working across 75 schools in the later one.)
In another study, Henry found that federally backed approach approach was somewhat more successful, leading to small test score increases.
“What seems to have been the case is that there needs to be a disruption in the status quo,” he said.
Still, it’s far from clear that researchers or policymakers know the best way to substantially improve a struggling school.
A national evaluation of the federal turnaround program found that it produced no clear benefits overall — though it did yield large gains in a number of specific states, according to other research. There is also some evidence that improvements were largest when schools fired not only the principal but half the teaching staff as well.
Meanwhile, a New York City program that flooded schools with extra resources and wraparound support increased student attendance but didn’t boost test scores or graduation rates. In Atlanta and Memphis, takeovers of low-performing schools by charter school operators have yielded uneven results; in Boston and New Orleans, a similar strategy has led to large gains.