Chalkbeat is a nonprofit news organization covering public education in communities across America. Sign up for our free weekly newsletter to keep up with how public education is changing.
Just as COVID hit some communities much harder than others, schools across the U.S. suffered disparate academic losses in the wake of the pandemic.
But new research points to a surprising finding: Students within the same district seemed to experience similar academic setbacks, regardless of their background. In the average district, white and more affluent students lost about the same amount of ground in reading and math as Black and Hispanic students and students from low-income families.
To researchers, that suggests that factors at the school district and community level — like whether students received quality remote instruction and whether communities experienced a strict lockdown — were bigger causes of test score declines than what was going on in students’ homes.
“Where children lived during the pandemic mattered more to their academic progress than their family background, income, or internet speed,” a team of researchers wrote in a report released Thursday.
The report offers some insight into why school districts experienced a wide range of academic losses during the pandemic. Citing pre-pandemic evidence that learning loss can persist for years without major interventions beyond normal instruction, it also points to the need for more intensive academic recovery efforts in some places. Those findings come as many schools are under pressure to reach more students with extra help like tutoring, and school leaders are trying to figure out the best ways to spend the limited COVID relief funding they have left.
But the report doesn’t get much closer to providing an answer to a key question that has evaded researchers: Why did school districts that stayed remote for similar lengths of time experience very different academic losses?
Thomas Kane, a Harvard professor of education and economics who co-authored the study, says that’s likely because researchers haven’t found a way to reliably measure factors that may have had a big impact, such as the quality of instruction students received.
“It’s like the suspect that we couldn’t find and question,” he said.
The team included researchers from Harvard, Stanford, Dartmouth, and Johns Hopkins universities, as well as the testing group NWEA. Together, they looked at data from 7,800 school districts in 40 states, focusing on reading and math scores from state and federal tests for students in third to eighth grades.
Then the team looked to build on earlier research released last fall that found academic losses were steeper in districts that served larger shares of Black and Hispanic students and students from low-income families, and in districts that stayed remote or offered a mix of in-person and virtual instruction for longer.
This time, the researchers looked at several more factors that they thought could have had an effect on student’s math and reading scores during the pandemic.
These included whether students had access to the internet and a device at home; school staffing levels; whether residents had trust in their local institutions, like schools; employment rates; COVID death rates; anxiety and depression rates; and the degree to which COVID caused social and economic disruptions in a community. (To identify those disruptions, the research team looked at how often people did activities such as shop for groceries, eat at a restaurant, or socialize with people outside their home, using a combination of cell phone, Google, and Facebook survey data.)
The team found that student test scores fell more, especially in math, in places where families saw their daily routines more significantly restricted — a finding that held true even in places where schools closed only for a short time. Math losses also were greater in counties that had higher death rates from COVID.
Meanwhile, learning losses associated with remote instruction were smaller in places that reported greater trust in their local institutions, perhaps because parents supported their local school district’s pandemic decision-making.
Math learning losses stemming from virtual learning were bigger in places where adults reported higher levels of anxiety and depression, and in communities that had higher employment rates. In those cases, researchers wrote, parents may not have been as able to support their kids when they were learning from home.
‘Extraordinary’ measures needed to help students recover academically
Still, the additional factors explain only a “little bit” of why academic losses varied so much in places that stayed remote longer, Kane said. And they don’t explain why high-poverty school districts that serve more students of color lost more academic ground when they stayed remote for longer.
That may be because researchers haven’t yet found a way to measure some of the most important factors. The team wasn’t able to look at community COVID hospitalization rates, for example. They also couldn’t take into account the quality of remote instruction students received, or what policies districts set for student attendance and engagement during remote learning.
Remote instruction varied widely, especially early in the pandemic. Some schools required students to attend classes on live video for several hours a day, while others gave students more independent work.
In some places, teachers received little training on how to teach students virtually. In other places, teachers had to juggle students who were both at home and in front of them — a setup that often left parents and students more dissatisfied with the instructional quality.
“In some schools remote instruction was a watered-down version of in-person instruction,” Kane said. “In other places, there was just much less of an expectation that classes would be covering the usual grade-level standards online. We just don’t have a direct measure of the quality of remote/hybrid instruction and the level of expectations.”
The researchers also found evidence that in the decade leading up to the pandemic, when districts saw big dips in test scores — perhaps because there was a strong flu season, or a weak teaching team that year — their students tended not to recover as they progressed through later grades.
That suggests, according to the researchers, that it will be difficult for students to recover from the pandemic unless their schools take “extraordinary” measures, like expanding summer school and tutoring many more students. Chalkbeat previously reported that in many of the nation’s largest districts, fewer than 1 in 10 students got any kind of tutoring earlier this school year.
“When there is a disruption, it’s not like they know how to hurry up,” Kane said. “They will proceed with their lesson plans and instruction. It’s easy to resume learning — it’s very hard to accelerate it.”
Kalyn Belsha is a national education reporter based in Chicago. Contact her at kbelsha@chalkbeat.org.
