While school leaders and lawmakers were quick to reason away concerns over shockingly low ILEARN scores, some testing experts and state education leaders are standing by Indiana’s new exam.
Calls to shield schools and teachers from any negative consequences of the low ILEARN scores were swift, after it was revealed that only one-third of students in grades 3-8 passed both the math and English portions of the exam. But when detangled from the question of accountability, experts say the results provide a valid measure of what students know.
Low 2019 scores weren’t a sign of a faulty exam, said Ed Roeber, Michigan’s former testing director and a consultant on Indiana’s technical advisory committee for assessments, said. Rather, Roeber said, it’s a reflection of “what instruction is or is not taking place in our schools.”
“I’m not discouraged by low performance,” he said. “I think it could be a real rallying cry for Indiana schools to evaluate what they are teaching and what students are learning.”
At the same time that State Superintendent Jennifer McCormick called for a one-year hold harmless in September — reiterating her concern over how test scores factor into the A-F grades the state gives to schools — she stood by the more rigorous test, noting the 2019 results align with how Indiana’s students perform in national measures.
“I think at the state level we’ve got to be realistic about our students,” she said at the time.
Since then multiple local leaders have also tried to bring public attention back to analyzing the results, and what they reveal. Claire Fiddian-Green, president and CEO of the Richard M. Fairbanks Foundation, said in an Indianapolis Business Journal op-ed that while ILEARN scores should be concerning, but that the more rigorous exam can help identify struggling students and schools. (Fairbanks Foundation, which supports educational initiatives, is a funder of Chalkbeat.)
In an op-ed for IndyStar, Brandon Brown, CEO of The Mind Trust, said the results show persistent racial gaps. While 43% of white students statewide passed both portions of ILEARN, 15% of black students did the same.
But some still have concerns about whether the results reflect how different the new test format is, rather than what students know.
“The results from that should be a baseline,” said Indiana State Teachers Association president Keith Gambill. “It is time now for educators to sit down with the results from that, now knowing how the test operates and how to best plan a course for students moving forward so in years to come the results are a true reflection of the growth of that student.”
Educator and parent concerns focused primarily on challenges associated with the new unfamiliar format, including issues with accommodations for students with disabilities. They questioned the varying amount of time it took students to complete the untimed test, and where the line determining if a student passes is set. Some wondered how so many fewer students were considered proficient this year compared to last.
Still, Roeber said ILEARN is a “good-quality test,” pointing to its clear blueprint that teachers helped create and its adaptive format, which allows students to receive easier or more difficult questions based on their previous answers.
The test blueprint ensures that students receive questions that address all of the state standards and that questions remain within a student’s grade-level, according to the Indiana Department of Education. Questions may get more or less complex, but they won’t jump ahead or drop behind a grade.
The test takes into account the difficulty of questions, so if both a high-performing student and a low-performing student each answer half of their questions right, the high-performing student would still receive a higher score because their questions were more difficult.
The adaptive format can offer a better measure of what a student knows than a scantron test, where every student gets the same questions, Roeber said. However, it could be more challenging for teachers to make sense of the results when not every student received the same questions.
Damian Betebenner, a consultant with the Center for Assessment who works with Indiana test data, likened questioning the tests validity when scores are low to throwing away your scale if you don’t like how much you weigh.
He said parents should consider both how much their child grew year-over-year in addition to whether they meet grade level expectations. Betebenner calculates the state’s growth scores, which measures how much students learned in a year compared to their peers. In general, student growth in Indiana has stayed steady, Betebenner said, so students who start behind stay behind and students who start above grade level stay above grade level.
The state could see better scores next year, as more students become familiar with the adaptive format. But Betebenner said he doesn’t expect to see a quick boost in scores, which states used to see in the second year of a new test.
Indiana never saw such an increase after scores plummeted with the new ISTEP test in 2015. Betebenner said that’s because tests are more rigorous, which means there are fewer shortcuts or tricks that students can use to perform better.
“You don’t get this bump of people catching on because the tests are actually quite rigorous,” Betebenner said. “The only things …going to help students get better scores on the test [is] mastery.”