Testing

New report shows IPS students lag on test improvement, but innovation schools may be a bright spot

PHOTO: Anthony Lanzilote

A new study finds mixed results for Indianapolis Public Schools dramatic shake-up in recent years: Students in schools within the district boundaries are below the state average when it comes to improvement on tests, but their peers at charter and innovation schools appear to be doing better.

Indianapolis Public Schools students are making smaller gains on math and reading tests than their peers across the state, according to a study released Thursday by the Stanford-based group CREDO, which looked at data from 2014-15 through 2016-17. It is the first in a series of studies examining 10 cities. In Indianapolis charter schools, students are about on par with peers across the state, researchers found.

“The pattern shows that overall Indianapolis schools do underperform the state as a whole during the study period. At the same time, we also noticed the difference by sector within the city,” said Chunping Han, a senior research analyst at CREDO who worked on the study.

The most highly anticipated part of the study is the first major look at the results for innovation schools, a new kind of district-charter partnership. Results from innovation schools show positive signs but left unanswered questions.

The study found that students at innovation schools, which were created in 2015-16 and have been rapidly expanding, made gains in math and reading in 2016-2017 that were similar to the state average. But the gains are not to a statistically significant degree.

If the innovation schools are able to maintain strong student improvement, it would be a remarkable boon for the district. The study is also further evidence that at least some of the innovation schools are helping students make big gains on state tests. When 2016-17 state test scores were released, several innovation schools had jumps in passing rates. But the inconclusive nature of the results also highlights how hard it is to judge a program that is still in its infancy.

Advocates for innovation schools nonetheless took the results of the study as support for the new strategy.

“We think this is an exciting indication that the work is leading to meaningful outcomes for students and is an initial validation of the innovation schools strategy as well,” said Brandon Brown, CEO of The Mind Trust, an Indianapolis nonprofit that supports innovation schools and provided technical assistance to CREDO with information on local schools.

A researcher who was not involved in the CREDO study cautioned that the results describe student test gains, but they do not explain why some students outperform others.

“I don’t think this analysis answers the question of ‘are Indianapolis Public Schools effective at boosting student achievement? Or are charter schools in Indianapolis effective at boosting student achievement?’ ” said Joseph Waddington, an assistant professor in the department of educational policy studies and evaluation at the University of Kentucky.

Waddington, who has co-authored research on the impact of Indiana’s school voucher program and school choice more broadly in Indianapolis, said that in order to draw conclusions about whether charter schools are responsible for larger test score gains, there would need to be “a more carefully constructed comparison group.”

Students who chose to attend Indianapolis charter schools, for example, may have access to far more school options than students in other parts of the state, he said.

“You are making these apples and oranges comparisons between students who have different access to schooling alternatives,” Waddington added.

CREDO typically matches students at charter schools with peers who attend nearby district schools — a method that has faced some criticism. In this case, the study compared students at charter, innovation, and Indianapolis Public Schools to students at schools that mirror the state average test score gains and demographics.

In a statement, Indianapolis Public Schools highlighted the positive results.

“The 2019 CREDO study confirms the exciting developments we’re seeing unfold in the classrooms of our innovation schools,” the statement read. “The positive academic outcomes are proof the strategies implemented by our talented school leaders are working. Likewise, we’re encouraged by strides we’re also seeing in our traditional schools as those outstanding leaders utilize increased autonomy to improve their growth.”

Since the district began creating innovation schools in 2015, their ranks have rapidly swelled. There are now 20 innovation schools, which enroll about one in four of Indianapolis Public Schools’ students.

Innovation schools have drawn national attention from advocates for collaboration between traditional districts and charter schools. They are under the Indianapolis Public Schools umbrella, and the district gets credit for their test results from the state. But the schools are run by outside charter or nonprofit managers. The network includes a variety of schools, including failing campuses that were overhauled with charter partners, new schools, and previously independent charters.

upheaval

Frustrations over principal turnover flare up at IPS School 43

PHOTO: Dylan Peers McCoy
School 43

It began with a tame slideshow presentation about hiring a new principal at School 43. But the Wednesday night meeting soon spiraled into a venting session — as parents and teachers pleaded with Indianapolis Public Schools to send in more staff.

Bakari Posey, the principal of School 43, departed for another job last week in the latest upheaval at the school, which is also known as James Whitcomb Riley. The assistant principal, Endia Ellison, has taken over in an interim capacity, as the district searches for a new leader for the school, which has faced significant turnover in recent years.

“This school needs help,” said Natasha Milam, who has three children at School 43, which serves about 450 students in prekindergarten to eighth-grade. “We need you all to listen. And we need you all to hear us.”

Milam, who volunteers at the school, said that because the building does not have enough staff to handle behavior problems, students are suspended far too often — meaning students are at home doing chores or getting into trouble, instead of in class learning.

Many in the neighborhood had hoped Posey, who is from the community, would be able to turn the school around after the previous two school leaders left their posts just months into the job. But under Posey’s leadership, the school continued to struggle on state tests, with just 7 percent of students passing both the math and English exams last year.

And after two-and-a-half years on the job, Posey left and began working this week as assistant principal at Fall Creek Valley Middle School in Lawrence Township. In an email Thursday, Posey said that he left because he thought the position in Lawrence would help him grow professionally and it was closer to his home.

Posey also disputed the picture of School 43 as a campus in crisis. He said this school year, there hasn’t been “turmoil in the school in regards to student behavior,” suspensions were down, and the campus has been “very calm.” (Suspension numbers could not immediately be verified.) He also said that Indianapolis Public Schools provided “great support” to school staff.

Nonetheless, parents and teachers’ at the meeting Wednesday said the school has serious problems.

Ryesha Jackson, a 4th-grade teacher who has been at the school a little over a year, said there are not enough staff to help with student discipline problems. That makes it hard for educators to teach, she said.

“We have fights almost every day,” Jackson said. “I guess my question is, ‘What are we doing right now to support teachers?’”

School 43 is a neighborhood school, on the north side of the district. More than 75 percent of students there are black, and almost 70 percent are poor enough to qualify for free or reduced-price meals — about the district average.

Indianapolis Public Schools interim Superintendent Aleesia Johnson said district and school leaders would work together to develop a plan to address the urgent problems at School 43.

“But what I can’t give you right now is the plan for that help,” she said. “That takes time and coordination with the school staff.”

The district is gathering input about what school community members are looking for in a principal before posting a listing, officials said. Finalists will be interviewed by committees of parents, community members, and school and district staff. The goal is to name a new principal by April.

Also at Wednesday’s meeting was a small contingent from the IPS Community Coalition, a group that is often critical of the Indianapolis Public Schools administration, particularly the district’s partnerships with charter schools.

Michele Lorbieski, a resident from the north side who ran unsuccessfully for the Indianapolis Public Board with the support of the coalition last year, said the district cannot just rely on the next principal to fix the school.

“What I’d hoped to hear tonight was what the school district was doing to put things in place to stop this revolving door of principals,” she said.

District officials did not directly address why turnover has been so high among principals at School 43. But Brynn Kardash, a district official who recently began working with the school, said that the central office is doing more to support it this year.

School 43 was added this year to the transformation zone — an effort to help troubled schools that includes dedicated support and regular visits from a team at the central office, said Kardash, the district’s executive director of schools for the zone. Educators in the zone get additional training, extra planning time, and help analyzing student data, she said.

“The goal is to really support Ms. Ellison in work that she’s doing,” Kardash said, “which then leads to, hopefully, teachers feeling that support in the classroom.”

technical difficulties

This personalized learning program was supposed to boost math scores. It didn’t, new study finds

PHOTO: Patrick Wall
A student at I.S. 228 in Brooklyn does online work through Teach to One, a program that grew out of the iZone.

A program that Bill Gates once called “the future of math” didn’t improve state test scores at schools that adopted it, according to a new study.

The research examines Teach to One, a “personalized learning” program used in schools across 11 states and which has drawn support from a number of major funders, including the Gates Foundation, Chan Zuckerberg Initiative, and Netflix co-founder Reed Hastings. (Gates and CZI are also funders of Chalkbeat.)

At five schools in Elizabeth, New Jersey, students who used Teach to One didn’t improve any faster than similar students who didn’t use the program, even after three years. The results underscore the limited evidence for claims that such technology programs can dramatically improve student learning, even as they have become magnets for philanthropic dollars.

“The original aspirations, that Teach to One programs were going to have huge positive effects on math scores — we can rule that out with these studies,” said Jonah Rockoff, a Columbia professor who studied an earlier iteration of the program.

Teach to One says its approach is designed to help students steadily learn math skills, regardless of how unprepared or advanced they are. Students spend time on a computer as well as with a teacher and working in small groups. Students receive individualized schedules each day based on their progress, and a computer program adapts the curriculum to students’ strengths and weaknesses in the form of a “playlist.”

New Classrooms, the organization behind Teach To One, suggests that the Elizabeth results aren’t the full story.

It points to a separate analysis released this week that looks at a broader group of schools — 14, from several districts — that used the program. That study shows Teach to One students making above-average gains on a test known as the MAP, which is taken on a computer with questions changing as students answer correctly or incorrectly.

New Classrooms co-founder Joel Rose suggested in a statement that those computer-adaptive tests capture something that state tests can miss: students’ progress.

“What seems to be emerging is a real tension in math between approaches focused on long-term academic growth and state accountability systems,” he said.

Rockoff said there might be something to New Classroom’s argument that the study using adaptive test is better able to showcase students’ gains. “If [students] are at a grade four level but they’re in grade six, teaching them grade four material is going to hurt them on the state test,” he said.

But the author of the second study, Jesse Margolis, and a number of other researchers who spoke to Chalkbeat note that it cannot show whether Teach to One caused any of the students’ gains, though — a major limitation.

“While this study cannot establish causality, it is encouraging,” Margolis wrote. (The New Jersey study is better able to establish cause and effect, but it also has limitations and does not rely on random assignment.)

The New Jersey study isn’t the first to show that Teach to One didn’t improve test scores: so did Rockoff’s 2015 report on three New York City middle schools that looked at both state and MAP tests.

One possible explanation is that Teach to One is helpful to students in some places but not others. Margolis said his study examined the same five Elizabeth schools as the Columbia study and also found minimal gains there, but that schools elsewhere seemed to see larger improvements.

Researcher John Pane of RAND, a leader in studying personalized learning, says the results are important to understanding a field with limited research to date.

“Because we have so little evidence on personalized learning,” he said, “every data point can be helpful for us to start triangulating and piecing together what works and what doesn’t work.”