Sometimes a bad grade is a good sign.
New research suggests that in New York City, giving schools a failing or near-failing grade reduced teacher turnover and helped attract more effective educators.
These results cut against the theory that low district-assigned grades would have precisely the opposite effect. They also provide new evidence that the now defunct A–F grading scale sparked improvements at underperforming city schools.
The study will be published in the peer-reviewed Journal of Human Resources. Because it focuses on comparing low-rated schools to those that received slightly higher grades, the study doesn’t offer a definitive verdict on how the grading system the city used from 2007 to 2014 affected the district as a whole.
What the findings do show: Schools that received a letter grade of D or F saw a 20 percent reduction in teacher turnover, compared to similar schools that barely scored one grade higher. The schools with lower grades also attracted higher-performing teachers, as measured by their ability to improve student test scores over time.
“This is an important counterpoint to the people who worry about the stigmatization effects of letter grades,” said Jonah Rockoff an education researcher at Columbia University, who reviewed the journal article and has studied the city’s school grading system.
And the results dovetail with Rockoff’s own previously published research showing New York City schools that received low grades went on to boost student achievement in reading and math.
Grades were based on student improvement on test scores over time, absolute student performance, and the school environment as measured by attendance and surveys. The grading system was at the heart of a sweeping set of reforms Mayor Michael Bloomberg’s administration implemented to give schools more autonomy, but also harsh consequences if they did not measure up.
Principals at top-scoring schools could become eligible for bonuses and extra funding, while bottom-ranked schools were subject to leadership changes or even closure. Critics feared that low grades would stigmatize schools and demoralize educators. And they voiced concerns that progress scores bounced around too much from year to year to be reliable. Echoing some of those worries, schools Chancellor Carmen Fariña, who retired in April, eliminated the system in 2014.
To measure the grading system’s impact during its first two years, the study uses data from 156 elementary and middle schools and compares those that barely received a D or F with those that scored just high enough to earn one grade higher. Lower ratings reduced teacher turnover, from about 11.5 percent of teachers leaving each year to 9 percent. That goes against both conventional wisdom and previous research from Florida, which also put in place letter grades for schools.
Reduced turnover could also signal that educators at low-performing schools had more trouble than their counterparts getting hired elsewhere. But Rebecca Dizon-Ross, the paper’s author and a professor at the University of Chicago, argues that it’s more likely that schools responded to the low grade by making improvements that encouraged teachers to stay.
Her study points out that teachers who attempted to leave low-rated schools were no less likely to be hired at other city schools, and lower-rated schools didn’t see fewer teachers transfer into them. Moreover, the teachers who did join schools after they received a failing grade tended to be more effective at improving test scores than teachers hired at top-ranked schools.
In addition, teachers at low-scoring schools tended to give their principals higher ratings after the poor grades became public — suggesting that school leaders made improvements in light of the added pressure from the district.
“If you’re at a school with a good principal that is responding well to a low accountability grade, that’s where teachers wind up staying,” Dizon-Ross said.
But the new findings come with some important caveats.
For one, each school’s letter grades were published after the school year started, making it difficult for teachers to leave immediately or mid-year, and potentially dampening any stigma effect.
And there is some evidence that not all bottom-ranked schools reduced turnover. Dizon-Ross’ data shows schools that received the lowest scores in the D or F range actually had higher turnover than schools that also received a D or F but were on the cusp of a higher grade. Her method focuses on schools that just missed the cutoff, so it can’t completely account for the experiences of schools that were the lowest performers.
Dizon-Ross acknowledged those limitations, but said her findings still show that the letter-grading system produced some important benefits, at least in New York City.
“The letter grading policy seemed to have this upside — it was designed to encourage low-performing schools to improve,” she said. “It’s a hopeful picture for accountability.”