Chicago Public Schools said Friday it will re-evaluate all the standardized tests it administers after its top investigator found irregularities in the administration of one high-stakes test that powers the city’s elementary school ratings.
Inspector General Nicholas Schuler said he found a dramatic difference between the amount of time Chicago students took to complete the untimed test known as NWEA/MAP during one testing cycle and the average time nationwide, discrepancies which could “be an indicator of cheating or of attempts to game the test,” according to a report provided to the Chicago Board of Education.
The district sent parents an e-mail Friday notifying them of the investigation and findings. It wrote that it would hire a national test security company to improve administration and security procedures and that the district would provide more thorough guidelines to schools, and oversee targeted audits in testing.
Nearly all district students take the NWEA at least twice a year starting in third grade, and often in earlier grades on some campuses. The results come back quickly and help paint a picture of a child’s academic growth. The collective scores contribute to nearly half of a school’s annual rating.
The test is also used for seventh graders’ applications for selective enrollment schools and in principal and teacher evaluations.
Speaking late Friday, Chicago Board of Education member Elizabeth Todd-Breland, an education researcher, said Schuler’s report was spurring top-level thinking about whether the district should continue to use the NWEA as a high-stakes test.
“I am very critical, frankly, about the way the NWEA is used, and I think we need to have that discussion.”
Schuler’s office reviewed nearly 321,000 tests taken by third through eighth graders in spring 2018. It found that 36% of tests took at least twice as long, and up to five times as long, as the national average. The report noted that Chicago was not timing the test the same way among schools, and that some students took three or four times as long to complete their tests, and paused them more often, as the average national rate.
“As a result, in some CPS schools and grades, a maximum 53-question test that the average student nationally completed in roughly an hour turned into a multi-day and even a weeklong event,” the inspector general wrote in a September memo to the Chicago Board of Education.
Schuler was looking into standardized testing following “numerous complaints to the OIG over the years about alleged NWEA cheating,” many based on anonymous complaints. The report offers little evidence of cheating, according to the school district.
He said he was surprised the report was released in advance of next week’s Board of Education meeting, the last one he will attend before stepping down from his post.
While acknowledging that the report provided 18 useful recommendations about proctoring and test administration that the district plans to adopt, Todd-Breland took issue Friday with the report’s methodology and use of the word “cheating.” She said it provided no evidence that educators were gaming the test and it did not offer correlating statistics that would back such a claim.
“When you actually run the data around both durations and pauses, there is no correlation between longer duration and more pauses and higher growth and achievement on the test,” she said. “There’s a mismatch between the evidence being provided and the assertions made in the report.”
It’s not clear what responsibility the testing company NWEA plays in the lapses. The testing company did not explicitly guide districts on imposing time constraints until December 2019, according to the district. Nor did it guide districts on when to allow use of the pause feature.
NWEA did not immediately respond to a message seeking comment.
Even as Chicago puts the NWEA under the microscope, New York City is moving toward expanding its use.
The pauses in testing could help students perform better. But they could also be abused, the inspector general said. “Students and teachers described a variety of improper practices, including gaming and cheating techniques, that could have added to test durations.
Chicago Public Schools has pointed with pride to its multiyear upward trend line on the NWEA, although scores flattened last year. Last spring, nearly 63% of elementary school students met national norms in reading and 57% did so in math.
District spokeswoman Emily Bolton said Friday that neither the inspector general’s report nor the subsequent district review should call into question the performance of Chicago students or individual schools.
“Our students — with the support of dedicated educators and principals — have made real and sustained academic progress over the past decade on a variety of assessments and metrics,” Bolton said in an emailed statement. “The review did not find any correlation between test duration and high academic growth and it does not call into question the accomplishments of our students and school communities.”
School administrators, teachers, and parents closely watch test scores. The NWEA/MAP is a key factor in how Chicago rates schools and in grade promotion. Students also take a separate, state-mandated standardized test. The new Illinois school superintendent, Carmen Ayala, is also reexamining assessments the state gives to its 2 million public school children.
Brentano Math and Science Academy teacher Aaron Bingea said schools take a long time on the NWEA test because they know the scores carry big consequences. For his seventh grade students, the NWEA is a significant portion of the grade that decides which high school they will be accepted to.
Getting a good score on the test involves extra work from both teachers and students, he said. Teachers have to teach material that is above the grade level of students because the computerized test adjusts the difficulty based on correct answers. Students will go slowly on the test, Bingea said, and will research content above their grade level.
Bingea, who presented his concerns about the NWEA to board members over the summer, suggested that the district remove the high-stakes nature of the test. “We are going to continue to put a lot of stress on kids, it will take a lot of time, and you are going to get a lot of bogus data” if the district continues to use the NWEA, he said.
Read the full reports here: