Enrolling in Success Academy boosted test scores for some of its earliest students, according to a new study conducted by research firm MDRC.
The analysis, funded by Success as part of a federal grant, found that the network’s third- and fourth-graders showed about a year to a year and a half of additional learning in math compared to similar students who also applied to the school, but lost the lottery. In reading, the gains are “positive,” though researchers caution they would need a larger sample to make a more definitive statement.
“The enrollment effects in math put Success Academy towards the top of what’s been estimated for charter schools,” said Rebecca Unterman, the author of the report.
The study is good, if relatively unsurprising, news for New York City’s largest charter network, which prides itself on sky-high test scores. When the state released test scores last week, Success Academy far outpaced the city, achieving an 84 percent pass rate in English and a 95 percent pass rate in math. Citywide, 40.6 percent of students passed English and 37.8 percent passed the math exam.
Critics have long contended that Success Academy achieves those results, in part, by pushing out the hardest-to-serve students. But those students are accounted for in this study, Unterman said. If a student started kindergarten at Success but did not stay through third grade, that student is still counted as “enrolled” for purposes of the study, she said.
Still, there are some caveats for these results. The study looks at students who enrolled in 2010, for instance, when there were only seven Success Academy charter schools. Now, the network operates 46 schools in New York City.
The study also includes statistics that will provide ammunition to critics of Success Academy. The report found that only about 50 percent of lottery winners chose to enroll in Success Academy schools. And applicants to Success were higher achieving than students in their surrounding neighborhoods.
Sixty percent of the control group members who did not win the Success lottery performed at or above grade level on their third-grade New York state math exams. Forty-four percent performed at or above grade level on their third-grade New York state reading exams. These percentages are at least 10 points higher than in the local district schools in the neighborhoods surrounding Success Academy schools.
In response, Success Academy officials suggested that the students denied admission may have gone to other charter schools, rather than district schools, which could have driven up their scores.
Those numbers call into question Success Academy’s frequent comparisons between its schools’ scores and citywide averages. Just last week, CEO Eva Moskowitz held a press conference and railed against the city’s test scores as proof of her network’s superior approach.
Rather than celebrating this study, Moskowitz said the results actually understate the network’s impact.
“MDRC studied Success results from students who entered our schools in 2010, seven years ago,” Moskowitz said in an email statement. “Our results are even better now.”