Mayor Bill de Blasio’s ambitious and expensive Renewal turnaround program appeared to push schools to improve attendance rates, but did not lead to significant gains on key academic metrics such as graduation rates or test scores, according to an internal education department report that has not previously been made public.
The report, prepared by the RAND Corporation, is the most rigorous study to date on the effectiveness of the mayor’s turnaround program, which cost $773 million and has been widely criticized for producing disappointing results. The city recently announced the program will not continue.
First some bright spots: The turnaround program resulted in students attending school more often. Renewal boosted daily attendance by 1.5 percentage points and led to a 5 percentage point reduction in chronic absenteeism, defined as missing 10 percent or more of the school year. The initiative also led students to accumulate more credits in high school, a sign that students were more poised to graduate on time.
But many of the report’s findings are not as positive, often pointing to only slight improvements that were not statistically significant. The program did not produce clear effects on graduation rates or test scores. Worse, the program slightly raised high school dropout rates. And the researchers, looking at variables such as curriculum, “high expectations,” and teacher teams and leadership, did not find any clear positive effects on instruction or school culture.
David Hay, the education department’s deputy chief of staff, said in an interview that officials found the results encouraging. “All of the sorts of things that we would look for to be improving at this moment in time we’re seeing improved,” he said, noting the report only looks at the first two full years of the four-year effort.
To isolate the effect of Renewal, the researchers compared schools in the program with other low-performing ones that fared just well enough to avoid being placed in Renewal. Differences between the schools could then be linked to the effects of the turnaround program, which gave schools a suite of additional social services and academic resources.
That means some of the lowest-performing schools were not included in the report, an important limitation. (Thirty-eight of the original 65 Renewal elementary and middle schools were included in the study; 26 out of 29 high schools were.)
In announcing the program, de Blasio said the city would “move heaven and earth” to improve 94 of the city’s lowest-performing schools. The schools, he said, would see “fast and intense” improvements within three years.
The RAND report suggests the schools did not meet those lofty expectations, but Hay and many of the program’s supporters argue the report’s results show an upward trajectory and that school turnaround can take several years. (Evidence of such a trajectory was seen in elementary and middle schools in the program but not high schools, according to the report.)
Hay emphasized that the city is not dialing back the help the current 50 Renewal schools were offered. And many of the program’s components — longer school days, extra counselors, dental clinics, and mental health support — were helpful to schools.
At a conference Friday, Hay said he believed Renewal had the right components but acknowledged some of the political headwinds.
“The thing that didn’t make sense is you got three years to make a big difference; we know that you’re generally not going to show results from a significant turnaround in that short of a time period,” Hay said. “That was part of the politics that led to some dysfunction later on. And the mayor that ran against closing schools, then found himself closing schools.”
Of the original 94 schools, 14 have been closed, nine have left the program after being merged with other schools, and city officials said 21 have shown sufficient progress to slowly ease out of the program.
Still, the city ultimately decided to wind the program down after this school year and has been reluctant to release the RAND report’s findings, some of which were first reported in the New York Times. Education department officials have refused to make the report public and repeatedly missed their own deadlines to provide it after Chalkbeat filed a request in October under the state’s open records law.
On Friday evening, the department complied with a records request once the report appeared on RAND’s website.
So why didn’t the department release findings they viewed as positive sooner?
Hay said the report was commissioned for internal use only, and department officials were bound by a contract with RAND not to distribute it publicly.
The findings confirm previous research that have found the Renewal program only produced modest effects, or none at all — something often seen in turnaround efforts nationally. But the RAND study, commissioned by the education department, is one of the most comprehensive looks to date at the New York City program.
You can read the full report below:
Matt Barnum contributed to this story.