Questar’s unauthorized change of an online testing tool — not a possible cyber attack, as earlier reported by the company — was responsible for shutting down Tennessee’s computerized exams on their second day this spring, the state’s chief investigator reported Wednesday.
An independent probe determined that “there was no cyber attack,” nor was any student data compromised, when thousands of students could not log onto the online exam known as TNReady on April 17.
Instead, investigators said, Questar was mostly responsible for this year’s testing miscues. The main culprit was a combination of “bugs in the software” and the slowness of a computerized tool designed to let students turn text into speech if they need audible instructions.
Comptroller Justin P. Wilson reviewed early findings of his office’s internal review and the external investigation by a company hired by the Education Department during a legislative hearing in Nashville.
Education Commissioner Candice McQueen also told lawmakers that Tennessee is docking Questar about $2.5 million this year out of its $30 million contract because of the online problems that plagued many students and schools during the three-week testing window.
Payments being withheld are punitive, as well as to cover the state’s costs to address the problems, she said, adding that other discounts could follow.
Last week, McQueen announced that the state plans to launch a new search this fall for one or more testing companies to take over TNReady beginning in the 2019-20 school year. She said a track record of successful online testing is a must.
The text-to-speech tool worked fine last fall when a smaller number of high school students tested online. But the state said Questar made a “significant and unauthorized change” to that feature before the launch of spring testing that affects the vast majority of Tennessee students.
“We now know this decision led to the severity of other issues we experienced during online testing,” the Education Department said in a statement.
House Speaker Beth Harwell and Rep. Jeremy Faison asked the comptroller to review the state’s contract with Questar, particularly related to reports of a possible cyber attack. Wilson’s office also looked into other technical snafus that disrupted student testing for days, prompting the legislature to pass emergency laws that make this year’s scores inconsequential.
“We believe that the student testing issues occurred primarily because of how Questar set the student assessment system up to work,” said Brent Rumbley, the comptroller’s information systems audit manager.
On the second day of exams, Rumbley said, those issues manifested themselves in a suspiciously high volume of internet traffic to the testing platform.
“That’s what led the Department of Education and Questar to believe that there may have been a cyber attack,” he told lawmakers. “This traffic eventually shut the system down.”
Even though Questar upgraded the processing capability of its equipment in response, students and educators continued to report problems logging in, staying online, and submitting tests until Questar turned off the text-to-speech tool beginning May 1.
The comptroller’s office also found that Questar was ill-prepared to handle the fallout from the technical glitches. For instance, the company struggled to manually recover the high number of tests that students couldn’t submit online. And school personnel calling the customer service line experienced wait times as long as 60 minutes, prompting many to just hang up.
New details emerged Wednesday about other testing problems, too.
On April 25, a Questar employee “inadvertently overrode” custom rosters statewide that allowed schools to match students with available testing devices. “As a result, teachers and test coordinators had to scramble to get students the tests they should take,” Rumbley said.
The next day, more problems erupted when an internet cable was severed by a dump truck in a traffic accident in Hawkins County.
“According to the vendor that manages the fiber optic line, 21 districts were without internet from approximately two to four hours,” said Rumbley, adding that neither Questar nor the department could have prevented the outage that day.
Lawmakers will get an expanded look at the Education Department and its testing program in November when Wilson’s office presents the results of a year-long performance audit, along with findings from a massive survey of Tennessee educators about TNReady.
The two-hour hearing gave lawmakers a platform to take jabs at McQueen and her department for their handling of testing.
Rep. Bo Mitchell admonished the Education Department for tweeting on the second day of testing that Questar “may have experienced a deliberate attack” that morning.
“This gets into the public trust and throwing out information to the public from the Department of Education that the failure was a hack … Whose decision was that to put that out into the public domain without any proof?” asked Mitchell, a Democrat from Nashville.
McQueen clarified that the department never used the word “hack,” but reported that the testing system was experiencing a “pattern of data that was consistent with a cyber attack.” The description was based on what was known as the time, she said.
Sen. Janice Bowling, a Republican from Tullahoma, said Questar’s $2.5 million penalty “seems like a smack on the wrist” given the disruption caused by the company’s mistakes.
McQueen responded that the state is withholding almost $11 million invoiced by Questar for online testing as it continues negotiations. She added that the state’s biggest testing expenses stem from printing and transit costs for paper materials used by about half of its students this year. The state is transitioning to computerized testing and has decided to slow the switch for a second time in the wake of this year’s challenges.
Questar officials told Chalkbeat last week that the company plans to pursue the state’s new contract next year, but Rep. Craig Fitzhugh told McQueen that he doesn’t want the Minnesota-based company involved after it completes its current contract.
“I don’t think we can let Questar get in the ballgame again,” said the Ripley Democrat.
The proposal will be competitively bid, said Wilson, adding that Questar’s past performance will be taken into account.