More than scores

New research takes an in-depth look at Chicago charter schools — and finds good news beyond test scores

PHOTO: Creative Commons / Charles Wiriawan

Most of what we know about the effectiveness of charter schools has come from scrutinizing testing data. That’s frustrated some who see the focus on test scores as too narrow. Now, a major study recently released by the University of Chicago Consortium on School Research joins a growing body of research that moves beyond tests to judge charters.

This analysis focuses on charter high schools in Chicago, which now educate over a fifth of Chicago high school students. The main takeaways: charter schools seem to help students in the short- and long-run, but those schools also have higher student turnover.

“Test scores are important, but so are other things,” said Julia Gwynne, one of the study’s authors.

The study’s findings will provide grist for both supporters and critics of charter schools. It also offers a number of important takeaways for national charter school observers.

The study shows that one of the larger charter high school sectors in the country is having big positive effects on students.

Attending a charter high school in Chicago led to substantial improvements in test scores, high school attendance, college enrollment, and college persistence.

These effects were relatively big: charter students were in attendance about eight more days on average and scored a full point higher on the ACT (which is out of 36 points). They were nearly 20 percentage points more likely to enroll in a four-year college, and also much more likely to persist in college through four semesters.

“The college enrollment rates are quite high – I would put that at a very sizable difference,” said Gwynne.

Source: UChicago Consortium of School Research

(It’s worth pausing here to note that although charters had a big edge in getting students to college, the college persistence for both sets of students was fairly low, echoing a national trend.)

These are not just raw comparisons of performance: the authors control for a variety of factors that might affect student performance in high school, including poverty, eighth grade test scores and attendance rates, and special education status. This strongly suggests that differences between students are caused by the quality of their schools. The study largely uses data between 2010 and 2013, although the paper goes further back to assess college outcomes, focusing on students who entered high school between 2008 and 2010.

These results are consistent with another recent study showing that Noble, a large charter network in Chicago, led to a big boost in college attendance.

On one important measure charter students were on par with district kids: high school graduation rates were essentially identical.

Interestingly, the results from this study are quite similar to research on Boston’s charter schools. Students there saw big gains in test scores and four-year college enrollment, but the same or lower high school graduation rates.

It’s among the first studies to document a common criticism lobbed at charter schools: high rates of student attrition.

There’s a big caveat to these findings: students are much more likely to leave a charter high school than a district one.

The natural question when considering the two main findings from the study — better outcomes but higher transfer rates — is whether one causes the other. That is, do Chicago charters just get better results because only more successful kids stick around? No, at least not directly.

To measure the effects of charters, the study follows students over time. Charter schools are judged by the performance of all students who start ninth grade at a charter — even if they subsequently transfer out. This is a common technique among researchers and means that the estimated charter impacts are likely conservative.

Still, the paper adds weight to a criticism that has dogged charters across the country: that they push kids out who are struggling or who have behavior problems. There have been a number of anecdotes to support this, but previous studies in a few districts, including New York City and Denver, had generally found little evidence that students leave charters at high rates.

Generally, low-achieving students in Chicago charters were more likely to transfer out than high-achieving ones — but that pattern was also true in district schools. Charter high schools had high attrition rates among both high- and low-performing kids.

The research does not show why students were more likely to leave charter schools.

Source: UChicago Consortium of School Research

The study highlights that even in a successful charter sector, there are some really low-performing schools.

It’s often heard in the charter school debate: charter success varies significantly from school to school. This is hardly surprising; it’s true of district schools too. But the Chicago study finds that on measures like test scores and college enrollment, charters vary even more than district schools in their impact on students.

That means there are more really good charter schools, but also more bad ones, even as on average the charters are better. (There is some evidence that is true of charter schools in Arizona, North Carolina, and Texas.)

Source: UChicago Consortium of School Research

The findings suggests that policymakers should be wary of judging a school by its sector, and that they need to be especially vigilant for struggling charter schools.

The study hints at — though doesn’t definitively explain — why some charter schools are successful.

The latest research can’t show why, overall, Chicago’s charter high schools seem to be high-performing, but it can point to ways they differ from district schools.

Charter high school teachers reported a higher sense of trust and collective responsibility among colleagues. Charter students said their schools were more engaged in helping them plan for life after high school, and teachers said there were greater expectations for college attendance. The schools also had tougher requirements for moving on to the next grade and for graduating high school.

Perhaps surprisingly, charters had a similar number of school days as the district.

Charter schools may have certain advantages over district schools. For instance, charter teachers report that parents are significantly more involved in schools; this may be a reflection of how charters work with parents, but it could also be about the families who select charters (or both).

The fact that charter schools have higher transfer rates may also matter, as does the finding that those students usually end up in district schools. Other research has shown that students entering school mid-year can hurt the performance of their peers, which might be a particular challenge for Chicago’s neighborhood schools.

The study does not look at financial differences across the sectors, including the role of outside money in supporting charter schools, nor does it examine school discipline or student suspension rates.

“What our next steps should be — really understanding best practices in high-performing charter schools is probably at the top of the list,” said Gwynne, the researcher. “To really understand what’s happening on the ground, you need to send researchers into schools.”

Regrouping

After another bumpy testing year, Tennessee likely will slow its switch to online exams

PHOTO: Marta W. Aldrich
Members of Tennessee's testing task force listen to a presentation by Mary Batilwalla, deputy commissioner over assessment for Tennessee's Department of Education. The group offered feedback on options for transitioning to online testing after more problems occurred this year.

Tennessee education leaders are rethinking their timeline for adopting computerized testing after a parade of technical problems bedeviled students taking the state’s TNReady exam for a third straight year.

Most students are scheduled to test online next school year under a three-year transition plan. But since keyboard testing had significant challenges this year with half that number of students, Education Commissioner Candice McQueen is backing off from that timetable.

And while there’s disagreement over exactly how to move ahead, there’s consensus about one thing.

“We have a credibility issue,” said state Rep. John Forgety, “and we need to get it right one time.”

McQueen floated three options for the 2018-19 school year to members of her testing task force during its Wednesday meeting in Nashville:

  •     Returning to paper testing across all grades for one year;
  •     Computer testing for high school students; paper testing for grades 3-8;
  •     Computer testing for grade 6 through high school; paper testing for grades 3-5

Off the table, however, is the option that districts had this year to give computer tests to more grades than required by the state.

The state ordered that all high school students take the test by computer this year, but about 40 percent of districts also chose to go digital for at least some of their students in grades 5-8.

The early thinking had been that letting districts test more students than required would expedite Tennessee’s online switch if local leaders felt ready. But state officials now believe the piecemeal approach only complicated the process.

“We feel very strongly” about this decision, Deputy Education Commissioner Mary Batilwalla told the task force. “The complexity is really too great for us to overcome in ensuring that we have a seamless delivery.”

The 30-member task force of educators and advocates has been McQueen’s sounding board on TNReady and other testing issues, and she sought the group’s feedback one week after the state’s messy testing season ended.

“We don’t want to introduce any additional complexity. We want to eliminate complexity, eliminate risk,” said McQueen, who also is turning to superintendents and upcoming focus groups for advice about how to improve their TNReady experience.

PHOTO: Marta W. Aldrich
Education Commissioner Candice McQueen speaks at a 2017 event as Gov. Bill Haslam looks on.

McQueen will decide about digital vs. paper — and for which grades — by late June. She is leaning toward keeping high schools online and putting all lower grades on paper tests, but it’s not a done deal, she told Chalkbeat.

“The feedback we’re getting is for more to go online than not, and that’s very meaningful to hear,” she said.

Her boss, Gov. Bill Haslam, has made it clear that Tennessee is committed to eventually adopting computerized testing.

“It’s not just that’s where the world is going; that’s where the world is,” Haslam said earlier in the week.

About 300,000 students took TNReady online this year — the most ever since a wholesale switch to computers failed in 2016 under Measurement Inc. McQueen fired that testing company, hired Questar as its successor, and unveiled a new game plan to gradually wade back in. That approach worked well last year for the 24 districts that did a trial run for high schools, although later scoring errors detracted from Questar’s debut.

This year marked the return to statewide online assessments, beginning with Tennessee’s oldest students. But challenges included a cyber attack and lousy internet service when a dump truck cut a main fiber optic cable — examples that demonstrate the risks of computerized testing.

There are benefits, too, however. Digital exams are quicker to score, offer more flexibility in the types of questions asked, and ultimately cost less. Returning to all paper testing would cost an extra $11 million in printing and shipping costs.

One big advantage of paper-and-pencil testing is a shorter testing period. Three weeks were allotted to TNReady this spring because schools had to rotate their students in and out of testing labs to use a limited number of computers. That requires a lot of coordination and time.

Task force members agreed that reverting to paper would be a step backward, especially with the state’s focus on the technical skills needed for college and careers and the significant investments made by school districts to prepare for online testing.

But they were adamant that Tennessee needs a win next time around to rebuild trust in a test that many consider broken.

“There has been a serious erosion in confidence in state testing, whether it’s online or on paper,” said Shawn Kimble, director of schools in Lauderdale County. “If we fail again, where does that leave us as a state?”

course count

In New York, students of color lack access to advanced coursework, new analysis finds

PHOTO: Emilija Manevska
Student writing on blackboard

If a student lived in a suburban, wealthy school district in New York state last year, her chances of attending a school with six or more Advanced Placement or International Baccalaureate were greater than 90 percent.

In New York City – where students are far more likely to be black and Hispanic – a student’s chances of accessing such a rich curriculum plummeted to 18 percent.

That is just one example of how New York’s black and Latino students are denied access to advanced coursework, including math, science, music, and foreign language classes, according to a new analysis of 2017 data released by the New York Equity Coalition, a group of about 20 civic organizations. The lack of access cripples students trying to prepare for college and denies them the chance to take rigorous coursework, the report’s authors argue.

“It should be cause for alarm and action,” said Ian Rosenblum, executive director of The Education Trust-NY, which is part of the coalition and conducted the analysis. “We see this question of access to rich and robust coursework as being essential for New York students.”

In New York City, officials have acknowledged many students of color lack access to advanced courses — and Mayor Bill de Blasio has vowed to fight it. The city has announced initiatives aimed at expanding middle school algebra courses, Advanced Placement classes in high school, and computer science education.

But Education Trust-NY’s new analysis sheds light on the depth of the problem facing the city. It also suggests that simply adding classes will not be enough to enroll more black and Latino students in advanced coursework, since these students are often under-enrolled in these courses even when they are offered at their schools.

The analysis looked at “gatekeeper” courses, which authors say either provide a springboard to higher-level courses or allow students to develop important skills or passions. The courses include middle school algebra and earth science, calculus, physics, Advanced Placement and International Baccalaureate classes, computer science, advanced foreign language, and music.

Across the board, Education Trust-NY found that students of color in New York state are under-enrolled in these courses compared to their white and Asian peers. For instance, for every 100 New York high school students, about 15 white students and 20 Asian took physics, while only around seven black or Latino students did the same.

The analysis finds two reasons for the lack of enrollment among New York’s black and Latino students. The first is that students of color are more likely to attend schools that do not offer these courses. That problem was particularly acute in the state’s urban centers, including New York City, which enroll a greater share of the state’s black and Hispanic students than other areas in the state.

For instance, the share of New York City schools that offered algebra in middle school and physics, calculus, music, and advanced foreign language in high school was more than 20 percentage points lower than the state average in each case.

Secondly, even if advanced courses are offered in schools, black and Latino students may not enroll in the classes, the analysis finds. For instance, in New York City, 56 percent of students in schools offering calculus last year were Latino or black, while only 35 percent of student enrolled in calculus were Latino or black.

The city is working hard to combat both of these problems, officials say. Since the mayor unveiled his “Equity and Excellence” agenda in 2015, 152 high schools are offering new Advanced Placement courses, teachers have been trained across 550 schools to offer computer science classes, and teachers across 357 elementary schools have received training in the city’s initiative to boost algebra participation.

Additionally, since the 2017 school year, which is the year used in Ed Trust-NY’s analysis, 89 more schools in New York City offer additional Advanced Placement classes, according to city officials. However, it is unclear exactly how many new schools are offering algebra in middle school or computer science classes, they said.

The city also instituted a Lead Higher initiative, aimed at reducing disparities in enrollment among underserved students at schools that already have AP classes.

However, there are some aspects of the city school system that might work against offering more advanced classes in every school. The previous administration split many large, comprehensive high schools into smaller schools. Since smaller schools may lack the teaching capacity or number of students to justify a wide range of courses, students’ options may be limited.

New York City’s high school system is also extremely stratified by academic achievement. Top schools are allowed to select the city’s high-performing students, while the remaining schools have few students who can complete grade-level work in English and math. As a result, those schools – which disproportionately serve students of color – may lack advanced classes.

Critics may say that the lack of advanced classes is a symptom of a bigger problem: That many black and Latino students have not been prepared for more advanced coursework in their elementary and middle schools. Rosenblum said that may be true in some cases, but there are also many students who are prepared to succeed in advanced classes but are not given the opportunity.

“The research is really clear that vastly more students can succeed in higher-level and advanced courses than are currently in them,” Rosenblum said, adding, “If we want students to be prepared for rigorous courses in high school, we need more rigorous courses to prepare them.”

The analysis also points to another reason that student of color may not be encouraged to pursue advanced coursework: a lack of guidance counselors. Eight percent of black and Latino students attend a middle school without a guidance counselor, which is double the rate of their white peers. In high school, about 40 percent of black and Latino students attend schools where there are more than 250 students for every guidance counselor, whereas 27 percent of their white peers do the same.

Rosenblum and others at the New York Equity Coalition have posited several solutions to the problems outlined in their analysis. One suggestion would have students default to a more advanced set of courses that begins with taking algebra in middle school. In this scenario, parents would have to sign a waiver to opt students out of this more challenging path.

Solutions like this have the potential to appeal to those with dueling educational philosophies. On the one hand, it could appeal to those who have been calling for higher educational standards – since it would encourage more advanced coursework. On the other, it does not rely on test scores to achieve those higher standards.

This debate has bubbled to the surface recently in a conversation about New York’s Regents exams, which students typically must pass before graduating. Some argue the tests help make graduation requirements more rigorous, while others say they are a poor way to ensure more students are prepared for college.

State policymakers have signaled they are interested in rethinking graduation requirements and have already carved out exceptions for some students that stray from the traditional path of passing five Regents exams. But they have not yet coupled it with a way to ensure that students remain focused on advanced classes, raising concerns from advocates that they have been dropping standards.

Further, a wide range of politicians and policymakers have called for increased access to rich coursework, including officials at the state education department, de Blasio, Gov. Andrew Cuomo and his Democratic primary rival Cynthia Nixon.