More than scores

New research takes an in-depth look at Chicago charter schools — and finds good news beyond test scores

PHOTO: Creative Commons / Charles Wiriawan

Most of what we know about the effectiveness of charter schools has come from scrutinizing testing data. That’s frustrated some who see the focus on test scores as too narrow. Now, a major study recently released by the University of Chicago Consortium on School Research joins a growing body of research that moves beyond tests to judge charters.

This analysis focuses on charter high schools in Chicago, which now educate over a fifth of Chicago high school students. The main takeaways: charter schools seem to help students in the short- and long-run, but those schools also have higher student turnover.

“Test scores are important, but so are other things,” said Julia Gwynne, one of the study’s authors.

The study’s findings will provide grist for both supporters and critics of charter schools. It also offers a number of important takeaways for national charter school observers.

The study shows that one of the larger charter high school sectors in the country is having big positive effects on students.

Attending a charter high school in Chicago led to substantial improvements in test scores, high school attendance, college enrollment, and college persistence.

These effects were relatively big: charter students were in attendance about eight more days on average and scored a full point higher on the ACT (which is out of 36 points). They were nearly 20 percentage points more likely to enroll in a four-year college, and also much more likely to persist in college through four semesters.

“The college enrollment rates are quite high – I would put that at a very sizable difference,” said Gwynne.

Source: UChicago Consortium of School Research

(It’s worth pausing here to note that although charters had a big edge in getting students to college, the college persistence for both sets of students was fairly low, echoing a national trend.)

These are not just raw comparisons of performance: the authors control for a variety of factors that might affect student performance in high school, including poverty, eighth grade test scores and attendance rates, and special education status. This strongly suggests that differences between students are caused by the quality of their schools. The study largely uses data between 2010 and 2013, although the paper goes further back to assess college outcomes, focusing on students who entered high school between 2008 and 2010.

These results are consistent with another recent study showing that Noble, a large charter network in Chicago, led to a big boost in college attendance.

On one important measure charter students were on par with district kids: high school graduation rates were essentially identical.

Interestingly, the results from this study are quite similar to research on Boston’s charter schools. Students there saw big gains in test scores and four-year college enrollment, but the same or lower high school graduation rates.

It’s among the first studies to document a common criticism lobbed at charter schools: high rates of student attrition.

There’s a big caveat to these findings: students are much more likely to leave a charter high school than a district one.

The natural question when considering the two main findings from the study — better outcomes but higher transfer rates — is whether one causes the other. That is, do Chicago charters just get better results because only more successful kids stick around? No, at least not directly.

To measure the effects of charters, the study follows students over time. Charter schools are judged by the performance of all students who start ninth grade at a charter — even if they subsequently transfer out. This is a common technique among researchers and means that the estimated charter impacts are likely conservative.

Still, the paper adds weight to a criticism that has dogged charters across the country: that they push kids out who are struggling or who have behavior problems. There have been a number of anecdotes to support this, but previous studies in a few districts, including New York City and Denver, had generally found little evidence that students leave charters at high rates.

Generally, low-achieving students in Chicago charters were more likely to transfer out than high-achieving ones — but that pattern was also true in district schools. Charter high schools had high attrition rates among both high- and low-performing kids.

The research does not show why students were more likely to leave charter schools.

Source: UChicago Consortium of School Research

The study highlights that even in a successful charter sector, there are some really low-performing schools.

It’s often heard in the charter school debate: charter success varies significantly from school to school. This is hardly surprising; it’s true of district schools too. But the Chicago study finds that on measures like test scores and college enrollment, charters vary even more than district schools in their impact on students.

That means there are more really good charter schools, but also more bad ones, even as on average the charters are better. (There is some evidence that is true of charter schools in Arizona, North Carolina, and Texas.)

Source: UChicago Consortium of School Research

The findings suggests that policymakers should be wary of judging a school by its sector, and that they need to be especially vigilant for struggling charter schools.

The study hints at — though doesn’t definitively explain — why some charter schools are successful.

The latest research can’t show why, overall, Chicago’s charter high schools seem to be high-performing, but it can point to ways they differ from district schools.

Charter high school teachers reported a higher sense of trust and collective responsibility among colleagues. Charter students said their schools were more engaged in helping them plan for life after high school, and teachers said there were greater expectations for college attendance. The schools also had tougher requirements for moving on to the next grade and for graduating high school.

Perhaps surprisingly, charters had a similar number of school days as the district.

Charter schools may have certain advantages over district schools. For instance, charter teachers report that parents are significantly more involved in schools; this may be a reflection of how charters work with parents, but it could also be about the families who select charters (or both).

The fact that charter schools have higher transfer rates may also matter, as does the finding that those students usually end up in district schools. Other research has shown that students entering school mid-year can hurt the performance of their peers, which might be a particular challenge for Chicago’s neighborhood schools.

The study does not look at financial differences across the sectors, including the role of outside money in supporting charter schools, nor does it examine school discipline or student suspension rates.

“What our next steps should be — really understanding best practices in high-performing charter schools is probably at the top of the list,” said Gwynne, the researcher. “To really understand what’s happening on the ground, you need to send researchers into schools.”

Mapping a Turnaround

This is what the State Board of Education hopes to order Adams 14 to do

PHOTO: Hyoung Chang/The Denver Post
Javier Abrego, superintendent of Adams 14 School District on April 17, 2018.

In Colorado’s first-ever attempt to give away management of a school district, state officials Thursday provided a preview of what the final order requiring Adams 14 to give up district management could include.

The State Board of Education is expected to approve its final directives to the district later this month.

Thursday, after expressing a lack of trust in district officials who pleaded their case, the state board asked the Attorney General’s office for advice and help in drafting a final order detailing how the district is to cede authority, and in what areas.

Colorado has never ordered an external organization to take over full management of an entire district.

Among details discussed Thursday, Adams 14 will be required to hire an external manager for at least four years. The district will have 90 days to finalize a contract with an external manager. If it doesn’t, or if the contract doesn’t meet the state’s guidelines, the state may pull the district’s accreditation, which would trigger dissolution of Adams 14.

State board chair Angelika Schroeder said no one wants to have to resort to that measure.

But districts should know, the state board does have “a few more tools in our toolbox,” she said.

In addition, if they get legal clearance, state board members would like to explicitly require the district:

  • To give up hiring and firing authority, at least for at-will employees who are administrators, but not teachers, to the external manager.
    When State Board member Steve Durham questioned the Adams 14 school board President Connie Quintana about this point on Wednesday, she made it clear she was not interested in giving up this authority.
  • To give up instructional, curricular, and teacher training decisions to the external manager.
  • To allow the new external manager to decide if there is value in continuing the existing work with nonprofit Beyond Textbooks.
    District officials have proposed they continue this work and are expanding Beyond Textbooks resources to more schools this year. The state review panel also suggested keeping the Beyond Textbooks partnership, mostly to give teachers continuity instead of switching strategies again.
  • To require Adams 14 to seek an outside manager that uses research-based strategies and has experience working in that role and with similar students.
  • To task the external manager with helping the district improve community engagement.
  • To be more open about their progress.
    The state board wants to be able to keep track of how things are going. State board member Rebecca McClellan said she would like the state board and the department’s progress monitor to be able to do unannounced site visits. Board member Jane Goff asked for brief weekly reports.
  • To allow the external manager to decide if the high school requires additional management or other support.
  • To allow state education officials, and/or the state board, to review the final contract between the district and its selected manager, to review for compliance with the final order.

Facing the potential for losing near total control over his district, Superintendent Javier Abrego Thursday afternoon thanked the state board for “honoring our request.”

The district had accepted the recommendation of external management and brought forward its own proposal — but with the district retaining more authority.

Asked about the ways in which the state board went above and beyond the district’s proposal, such as giving the outside manager the authority to hire and fire administrative staff, Abrego did not seem concerned.

“That has not been determined yet,” he said. “That will all be negotiated.”

The state board asked that the final order include clear instructions about next steps if the district failed to comply with the state’s order.

Indiana A-F grades

Why it’s hard to compare Indianapolis schools under the A-F grading system

PHOTO: Dylan Peers McCoy
Because Thomas Gregg Neighborhood School became an innovation school last year, the state uses a different scale to grade it.

A-F grades for schools across Indiana were released Wednesday, but in the state’s largest district, the grades aren’t necessarily an easy way to compare schools.

An increasing share of Indianapolis Public Schools campuses, last year about 20 percent, are being measured by a different yardstick than others, creating a system where schools with virtually identical results on state tests can receive vastly different letter grades.

The letter grades aim to show how well schools are serving students by measuring both how their students score on state tests and how much their scores improve. But as Chalkbeat reported last year, new schools and schools that join the IPS innovation network can opt to be graded for three years based only on the second measure, known as growth. Schools in the innovation network are part of the district, but they are run by outside charter or nonprofit operators.

Of the 11 out 70 Indianapolis Public Schools campuses that received A marks from the state, eight were graded based on growth alone. They included a school in its first year of operation and seven innovation schools.

At the same time, traditional neighborhood and magnet schools with growth scores as good as or better than the scores at A-rated innovation schools received Bs, Cs, and even Ds.

Of the 13 innovation schools that received grades for last school year, eight received As, two got Bs, two got Cs, and one got a D. Only Herron High School was graded on the same scale as other schools. (For high schools, grades incorporate other measures including graduation rates.)

The result is a system that most parents don’t understand, said Seretha Edwards, a parent of four children at School 43, a school that received a failing grade from the state but would have gotten a B if it were measured by growth alone.

“I just think it’s kind of deceiving,” she added. “I don’t think it paints a fair picture of the schools.”

Indianapolis Public Schools deputy superintendent for academics Aleesia Johnson said the growth scores show schools are on a good trajectory.

“If you see that kids are making progress in terms of growth, that’s a good sign that you’re on the right track,” she said.

Still, she acknowledged that “there’s still a lot of work to do” to get students to pass tests and show proficiency.

Johnson pointed out that often-changing standardized tests and different A-F grades can cause confusion for families, and those measures don’t provide a complete or timely picture for families who want to assess their schools or choose new ones. “I don’t think it gives a lot of valuable information,” she said.

Advocates have said the growth only model makes sense because schools shouldn’t be held accountable for the low passing rates of students that they just began educating. But in practice, the policy benefits charter and innovation schools, which enjoy strong support from Republican lawmakers.

“The concept behind the growth-only model was that we measured newer schools based off of what they are able to do for their students, rather than taking them where they received them,” said Maggie Paino, the director of accountability for the education department. “You’re taking strides to get toward proficiency.”

The situation is even more muddled than usual this year. Schools across the state received two letter grades. One was calculated under a state model that relies largely on test scores, and the other was determined under a plan the state uses to comply with federal standards.

In addition to helping parents choose schools, years of repeated low letter grades from the state can trigger intervention or takeover. But the state has deferred in decisions about intervening in low-rated schools to IPS in recent years.

Back in 2012, the state took over four chronically low-performing Indianapolis schools. Since Superintendent Lewis Ferebee took over, IPS has taken aggressive steps to overhaul struggling schools by “restarting” them as innovation schools with new managers. Other struggling schools have been closed.

School 63, which received its sixth consecutive F from the state, might have faced state intervention in the past. But the school is unlikely to face repercussions because IPS restarted the school by turning it over to an outside manager. The Haughville elementary school is now managed by Matchbook Learning.

Shaina Cavazos and Stephanie Wang contributed reporting.