It's been a big year for the DOE. In September, it won the Broad Prize, given each year to an urban school district that has improved its poor and minority students' test scores. This spring, students continued on their upward trajectory, at least according to the state math and reading scores that were released yesterday. But the biggest coup may have happened this past weekend, when the DOE, in partnership with the agency Droga5, snagged a prestigious international advertising award given each year to the “most innovative and ground-breaking idea” in advertising. The DOE took home the Cannes Lion Titanium Award for the "Million" Motivation Campaign, which aims to increase students’ engagement with school through the use of cell phones. Through a partnership with Verizon and Samsung, the DOE gave cell phones to 2,500 students in seven middle schools. The number of minutes available to each student depended on their performance in school; a child who successfully completed all of his work, therefore, would have more minutes to use than a lackluster student. When the program launched last fall, the DOE planned to use the phones to deliver motivational text and voice messages, sometimes from celebrities such as Jay-Z; it’s not clear whether that portion of the campaign has been rolled out yet.
As the number of incentive programs has risen in New York City, so have the city's state test scores. But does that mean the two trends are related? If you believe the Post's headlines, then the answer is yes. Yesterday, the paper ran one article about test scores rising in virtually all schools where students are now paid for their performance and another about test scores inching up at the 158 schools where teachers are to receive bonuses if their students do well. Today, the Post reported that 23 of the 51 middle schools targeted for extra resources due to persistent low performance post higher-than-average gains. But you have to look no further than the Post's own coverage to see that it's impossible to determine whether any of the incentives programs have paid off. In two of the programs — teacher bonuses and the middle school initiative — more than half of participating schools saw below average improvement. The Post declares that the third, the controversial new program that pays some students in selected schools for particular successes, "dramatically improved test scores" because scores rose in almost all of the 35 schools included in the program. But here, as with all of the incentive programs, we know only about correlation, not causation. And as part of Opportunity NYC, the student-payment program included only some students at the eligible schools — a fact that suggests that some other force may also have been at play in those schools' test score jumps.
No one was surprised when Chancellor Klein announced today that the city's students posted dramatic gains on state test scores this year. Charting a clear trajectory of improvement has been fundamental to his reforms. This year, he announced, nearly 80 percent of 4th graders and 60 percent of 8th graders passed the state math test, and about 60 percent of 4th graders and 40 percent of 8th graders passed the state English test. Gains in the last six years, the DOE points out in its press release, range from about 15 points in 8th grade English to more than 30 points on math tests at all levels. Even before the mayor made his announcement this afternoon, discussion had begun over whether this year's test scores are a sign of victory, as the mayor believes, or of score inflation and manipulation. In today's Sun, Elizabeth Green speaks to statisticians who warn that, for many reasons, large-scale score increases are not always to be believed.