America’s public school classrooms are full of students who aren’t being challenged.
That’s the claim of a new report by TNTP, the nonprofit advocacy and consulting group, looking at student work and real-life teaching. Students are “planning their futures on the belief that doing well in school creates opportunities — that showing up, doing the work, and meeting their teachers’ expectations will prepare them for what’s next,” it says. “Unfortunately, it’s a myth.”
The study, called The Opportunity Myth, relies on TNTP’s exhaustive effort to get at what students are really doing in class by surveying them in real time, reviewing their work, and observing class instruction — a combination rarely seen in education research. Based on this data, the report argues that low-income students of color in particular are suffering through mediocre instruction and simplistic classwork while their teachers expect little of them.
“Students spent more than 500 hours per school year on assignments that weren’t appropriate for their grade and with instruction that didn’t ask enough of them,” the report says.
It’s not clear that the study’s methods can support such strong conclusions, though. TNTP’s claims turn on its own subjective way of rating instruction and assignments, and it’s unclear if different approaches would yield different results. And the paper examined just four districts and one charter school network, all anonymous.
That means the study is at once extensive and limited: extensive because it amounts to a massive undertaking to better understand students’ experience, but limited because it only examines a fraction of students in a fraction of classrooms in a handful of districts, none of which were chosen randomly.
Regardless of debates about the methods, the report may draw significant attention. The research of TNTP, previously known as the The New Teacher Project, has a track record of shaping policy, particularly with an influential 2009 report known as The Widget Effect, which focused on perceived flaws of teacher evaluation systems.
The latest study was funded by the Joyce Foundation, the Chan Zuckerberg Initiative, the Nellie Mae Education Foundation, the Walton Family Foundation, the Overdeck Family Foundation, and the Barr Foundation. (Chan Zuckerberg, Joyce, Overdeck, and Walton are also funders of Chalkbeat.)
In contrast with some of the group’s past work, the latest report concludes with few controversial policy recommendations, instead calling for higher expectations and a careful examination of disparities in school resources.
“We believe it’s time to move beyond important but narrow debates — from how to measure teacher performance to charter versus district to the role of standardized testing — and return to the basic guiding principle that brings us to this work: the right of every student to learn what they need to reach their goals,” the report says.
TNTP focuses on three large urban districts, one small rural district, and one charter network with three schools in separate cities. In all but the rural district, a majority of students are black or Latino, as well as low-income.
From there, TNTP got a handful of teachers from certain schools in each district to document their students’ work, collecting and photographing the assignments done by six students during three separate weeks. (Students had to receive parental consent and their names were removed from the work.) TNTP then assigned a rating to each significant piece of work, looking at whether it was on grade level, among other traits.
TNTP also had observers watch and then rate two lessons by each teacher using the group’s rubrics and surveyed teachers to determine their views on whether students could meet their state’s academic standards.
Finally, they surveyed students on their classroom experiences. TNTP used a novel approach for tracking student feelings, asking students whether they were bored or felt excited about learning at various points in a lesson.
In all, TNTP says, it reviewed nearly 1,000 lessons, 20,000 examples of student work, and 30,000 real-time student surveys. And the results, the report said, are grim: only 16 percent of lessons observed had “strong instruction,” and about a quarter of assignments were deemed “grade appropriate.”
This varied from district to district and classroom to classroom. In the most specific example provided in the report, one eighth-grade assignment asked a student to fill in the missing vowels from the word “habitat” after reading a short passage; in contrast, another required students to write a lengthy essay based on a memoir of one of the students to desegregate the all-white high school in Little Rock, Arkansas.
Student surveys were somewhat more positive: a narrow majority of students, about 55 percent, were generally engaged and interested during class, based on TNTP’s survey.
Students of color and low-income students tended to be in classes with worse instruction, fewer grade-level assignments, and lower expectations for meeting standards. That was correlated with slightly lower rates of test score growth.
All of that, TNTP concludes, amounts to a damning case against most of the classrooms in question and American schools in general. “Students spend most of their time in school without access to four key resources: grade-appropriate assignments, strong instruction, deep engagement, and teachers who hold high expectations,” the report says.
“The ‘achievement gap,’ then, isn’t inevitable. It’s baked into the system, resulting from the decisions adults make.”
But TNTP faces steep challenges in using its data to make such strong claims.
In addition to the districts, schools, teachers, and students not being chosen randomly, the report is not able to pin down whether those resources lead to higher achievement or definitively show why some students seem to have less access to the key resources it cites.
One of the report’s central claims, that increasing access to those resources will boost students’ academic performance, rests on relatively small correlations. In fact, the study showed little if any overall relationship between teachers’ observation scores and their effects on test scores.
“We need to be a little careful about asserting that by increasing one or more of the four resources we will necessarily improve outcomes for kids,” said Jim Wyckoff, a professor at the University of Virginia who sat on an advisory panel for the report, while also noting that he thought the basic theory of the report made sense.
The report’s appendix notes that “classrooms with initially higher performing students tended to get better assignments, better instruction, were more engaged, and had teachers with significantly higher expectations.” But other research has shown that observers tend to give unfairly high ratings to classrooms with more high-achieving students, meaning cause and effect could run the other way here.
TNTP’s measure of teacher expectations relies on teachers’ responses to statements like “My students need something different than what is outlined in the standards,” something that may be conflating high expectations with teachers’ views about the quality of their state standards.
Still, one of the main takeaways from the report — that low-income students of color have less access to good teaching — is generally backed by past research.
High-poverty schools have higher rates of teacher turnover and more inexperienced teachers, on average. Other research in a number of states and cities, including Washington state, North Carolina, New York City, and Los Angeles, has shown that teachers of low-income students are less effective at raising test scores.