A study that seeks to understand the effects of Denver’s education reform policies is moving forward — but not without significant pushback on whether researchers should have access to the student data that would allow them to answer key questions.
The disagreement highlights how politicized education research can be — even as access to data is critical to providing the information that might cut through the politics.
“Data are power,” said Katharine Strunk, dean of the University of Pennsylvania’s Graduate School of Education. “That is true in any walk of life. It has grown increasingly political as we have seen the general polarization around public education. It is weaponized, and it doesn’t need to be.”
Parker Baxter, who directs the university’s Center for Education Policy Analysis, plans to examine the academic growth and graduation rates of Denver students who attended schools that were closed for poor performance, new schools opened to offer better options, or schools that received district turnaround grants.
These education reform strategies were used in Denver from 2008 to 2019, when a union-backed school board took office and these policies fell out of favor.
To carry out the study, Baxter requested access to anonymized student data from Denver and 11 comparison districts. In Colorado, unlike many other states, the elected State Board of Education must sign off on such requests. Usually they are approved, but in this case, Denver Superintendent Alex Marrero opposed the request. It was the first time state officials have had to grapple with district opposition. No other district publicly objected to being included in the data set.
In June, a divided State Board of Education voted 5-4 to grant Baxter’s request. Board Chair Rebecca McClellan and member Angelika Schroeder, both Democrats, joined three Republicans to support releasing the data. Democratic board members Lisa Escárcega, Kathy Plomer, Rhonda Solis, and Karla Esser voted no after a nearly two-hour public hearing.
Education policy sometimes divides Democrats. The same 5-4 split has marked recent State Board decisions to grant a charter appeal and remove the Adams 14 district’s chartering authority.
The opposing board members argued that Baxter’s research questions were too narrow and that his conclusions might be limited. Education department staff had recommended approving the request because it might help inform state policy — reasoning that sparked even more concern for some board members.
Solis pointed to a case study of school improvement in the Greeley-Evans district where she previously served as a school board member. She thought the study failed to consider the community organizing that preceded school improvement efforts. Those efforts wouldn’t have been as successful without the community work that came first, Solis said.
“When you don’t have the whole story, then people can weaponize certain areas to say, ‘They did it this way,’” Solis said at the June meeting. “But did they really? Because there were all these other elements. My concern with the research is that it tells a narrative but not the whole narrative.”
Esser said Baxter’s study might attribute all the improvements to the reform strategies when other changes might have had greater impact. Denver increased per-student funding, reduced student-teacher ratios, and expanded training and collaboration during the same time period.
“We’re going to say these are the only changes we’re looking at, and then we’re going to say that it was the portfolio method that led to this improvement or it didn’t,” she said. The portfolio method or model is another term for Denver’s approach of supporting school choice and a range of school types.
Schroeder said she was “amazed” at her colleagues’ opposition.
“I’m not sure I’ve ever been part of a discussion about research where people were afraid of the results because of how they’ll be used,” she said. “That’s what seems to be going on here. Good research does little more than create new questions and you keep going.”
The study is a followup to one Baxter published earlier this year that attributed significant improvements in student test scores and graduation rates to education reform strategies. One criticism of the first study was that without student-level data, the study couldn’t determine which strategies actually made a difference and didn’t entirely account for how the large increase in white, more affluent students during the same time period may have affected test scores.
Baxter said he hopes the next study can answer those questions more definitively — and he can’t do it without student-level data. Baxter has been a supporter of education reform, but he said that won’t influence his findings, which he hopes to publish in early 2024.
“People are already talking about what they think the impact of the reforms were, but we have not had empirical evidence to inform that debate,” he said. “I have opinions on the reforms, and I have hypotheses, but I think I’m demonstrating my commitment to the facts by doing the research and being willing to publish the results” wherever they lead.
Denver dispute unusual in part because it was public
Politics sometimes seeps into education research. In 2016, Louisiana ended a data-sharing agreement with MIT and Duke University after researchers at the two universities published a study that showed negative outcomes in the first year of the state’s voucher program. The state superintendent said researchers should have given the program more time before publishing any findings.
But researchers told Chalkbeat the kind of public pushback that happened in Colorado, with elected officials questioning research methodology, is rare. More often, an education department or school district might slow-walk a request, charge large amounts of money for data, or say data aren’t collected in ways the researcher can use. Some institutions only provide data for studies that align with their priorities, researchers said.
“I think people are sometimes cagey about saying that they don’t want to provide data because of the nature of a research question,” said Dan Goldhaber, director of the Center for Education Data & Research at the University of Washington. “I don’t think we typically see it spill out into the open like this.”
Doug Harris, a Tulane University economics professor who heads the National Center for Research on Education Access and Choice, said the federal government could help by requiring states to make more data available while also providing money to support data collection and analysis. It’s not ideal for elected officials to make that call, he said.
Colorado’s system might allow for some transparency around decisions, he said, but could also have a chilling effect on researchers who watch the process and predict that politicians won’t like their research question.
The nature of a study like Baxter’s will address some of the concerns State Board members raised, Harris said. For example, if other schools or districts made changes that also helped student learning, the improvements at turnaround schools or at new charter schools will look relatively smaller. That’s the benefit of using a large set of individual student data and comparing data within Denver and across districts.
Strunk said she sees Michigan, where she previously was an education professor, as a model. The Michigan Education Data Center, created through a partnership between the state education department and its flagship public universities, serves as a central clearinghouse to clean and store data, review requests, and help researchers refine their proposals.
It’s true, she said, that research questions have to be carefully designed to not mislead policymakers. For example, it would be wrong to study Michigan’s policy holding back third-graders who are poor readers by comparing students who are retained with those who are not. Instead, a researcher would want to compare only students from similar backgrounds who were eligible for retention and look at outcomes for those who were and were not held back.
And because education policies almost always involve complex tradeoffs and conflicting values, politics will probably be unavoidable.
“It’s not just the fault of districts and state agencies,” she said. “It’s also the fault of researchers who are careless, when data is cherry-picked in certain ways. I don’t see a way out of it not being political unless you make it so dry, with a 20-year moratorium on using the data, and then it’s not useful.”
Bureau Chief Erica Meltzer covers education policy and politics and oversees Chalkbeat Colorado’s education coverage. Contact Erica at emeltzer@chalkbeat.org.
