The city’s school-support networks have had little overall impact on student achievement, failing to overcome the powerful link between students’ backgrounds and their academic performance, according to a new report. The report comes as Chancellor Carmen Fariña is set to overhaul the school-support system.
Under former Mayor Michael Bloomberg’s decentralized school system, schools received instructional and operational support from whichever networks they chose to join. The network a school is part of can predict to a small degree how well it will do academically, according to the independent report by researchers at Brown University’s Annenberg Institute for School Reform.
But when schools’ varying demographic data — students’ race and ethnicity, poverty level, or special needs — are held constant, the networks have “little, if any” effect on schools’ academic performance, the researchers found. Meanwhile, the makeup of a school’s student body remains a strong predictor of its academic success, the report adds.
“Demography still seems to determine destiny in the current, network-organized school system,” according to the study, which will be published next month and was first reported by the New York Daily News.
Annenberg sent a draft to reporters ahead of a speech by Chancellor Carmen Fariña Thursday, where she will announce details of a new school governance system. She is widely expected to dismantle most of the 57 support networks, in which case the report could help justify the shakeup.
The researchers looked at publicly available test scores and graduation rates from 2010 to 2012 for each networks’ schools. Using statistical analysis and modeling, they found that up to nearly 30 percent of the variation in academic outcomes among schools could be explained by which network a school belongs to. However, far more of the variation — up to 90 percent for middle score reading scores, for example — was associated with the student population that a given school serves.
The researchers also conducted another analysis where they measured how well a hypothetical school in each network would perform academically if its student population matched the citywide averages. When they controlled for demographics in that way, they found that networks had very little impact on elementary and middle school academics. (However, the same did not hold true for high school graduation rates.)
Instead, most networks in that analysis hovered close to the citywide average. The researchers said they could not find any “beat-the-odds” networks whose schools had academic results that significantly exceeded the average for all networks.
“We could not see, in the analysis we did, that the networks as a whole have any positive or negative effects on schools, especially when you compare that to the effects of demographics,” said co-author Norm Fruchter, who also is a member of the city’s educational oversight board, the Panel for Educational Policy.
The report offers a limited view of networks, which the authors acknowledge. It focuses narrowly on the average results of the networks’ schools in those three years, rather than trying to measure how students’ performance changed over time. It does not look at how individual schools fared under the network system, nor does it examine the practices of any particular network.
“It’s looking from 32,000 feet,” Fruchter said. “It’s not saying that there are no successful networks or networks that were successful with individual schools.”
The report, which Annenberg funded and conducted independently of the city, also looked at the student populations of the schools in each network.
It found wide variation, with some networks supporting more schools with high-needs students than others. However, it also noted that the networks each serve a greater mix of racial and ethnic student groups than the districts that preceded them. The report attributes this to the fact that, unlike districts, the networks are non-geographic and span multiple boroughs.
The report makes a few recommendations, including that struggling schools be given more intensive support than they have received under the network system. A 2013 study commissioned by the Bloomberg administration made a similar suggestion.
That report by the Parthenon Group did not analyze the overall impact of networks on students’ academic performance. However, it did say that surveys and interviews showed that many principals preferred the network system.
“When the DOE asks schools to rate their satisfaction with the service they receive from various central offices, the network teams are consistently rated far higher than other central supports,” the report said.
Update: Here is the full report.