Ninth-graders across Colorado took the PSAT test for the first time this spring, and the results for Denver students have left district officials disappointed and confused.
Whereas Denver ninth-graders historically scored well on previous state tests, their performance on the preparatory college entrance exam was surprisingly low. That was especially true in reading and writing for students identified as English language learners – a trend mirrored at the statewide level.
Officials don’t know why. Dissecting annual state testing data at a recent meeting, Denver school board members were pleased to see performance improving at the elementary and middle school levels, but were concerned about ninth grade.
“I’m having assessment blues,” said board member Happy Haynes. “We’ve had so much focus on trying to get assessments that are meaningful and that tell us the story about our kids’ proficiency.” Looking at the high school scores, Haynes said, “I’m just baffled.”
The statewide switch to the PSAT was meant to make annual state testing more meaningful to ninth-graders, enough of whom were opting out of the previous standardized tests that state lawmakers took notice. Colorado 10th-graders also take the preparatory PSAT, and 11th-graders take the SAT, which many colleges use to help determine admission.
The test’s higher stakes make it more troubling that students are doing poorly.
Because spring 2018 was the first time ninth-graders took the PSAT, it’s impossible to compare their raw scores year over year. But it is possible to compare students’ “growth scores,” a state calculation that is supposed to show how much students learn in an academic year.
To measure that, the state compares how each student performed on the test relative to other students with similar score histories. Students are given a score from 1 to 99. Those with scores above 50 are considered to have made more than a year’s worth of academic progress in a year’s time. Students with scores below 50 are considered to have made less than that.
The state also averages students’ scores to calculate growth for entire districts and schools, and for specific groups of students. In spring 2018, Denver ninth-graders earned a growth score of 37 in reading and writing. That was a precipitous 24-point drop from the growth score of 61 that ninth-graders had earned in spring 2017 on a different standardized test called PARCC.
The ninth-grade growth score also decreased in math this year, though not by as much.
The declines could perhaps be explained away by reasoning that the 2018 ninth-graders were entirely different students from the 2017 ninth-graders. But as Denver Public Schools Superintendent Tom Boasberg pointed out, other factors remained constant.
“Same classrooms, same teachers,” he said, referring to the Denver educators who teach ninth grade. “This year, different assessment – and their growth fell off a cliff.”
Even when the district compared the 2018 ninth-grade growth scores with the growth scores those same students earned in 2017 when they were eighth-graders, there was a big dip.
Boasberg called the results “puzzling.”
“There is lots for us to ask,” he said, “and lots for us to try to understand.”
Some other large school districts in the metro area did not see a decline with the switch to the PSAT. Ninth-grade growth scores in Jeffco, Cherry Creek, Douglas County, Colorado Springs, and Boulder Valley either stayed the same or went up from 2017 to 2018.
Those districts have fewer students from low-income families than Denver. They also have fewer students of color and fewer students for whom English is a second language.
A particularly unusual and alarming result for Denver Public Schools was the growth scores for that last group: students whose native language is not English. It’s a big population, with more than a third of the district’s 92,600 students classified as English language learners.
They include students who have “exited” or been “redesignated” because they achieved fluency in English. Such students have historically done well on state tests and had above-average academic growth. But that wasn’t true this year on the PSAT. Ninth-grade students in that category earned a growth score of just 35 in reading and writing.
“We want to make sure we’re working with our English learners and our redesignated students on the language of the assessment to ensure it’s not an issue of test bias,” Deputy Superintendent Susana Cordova told the school board.
The SAT tests have been criticized for being biased in myriad ways. (So, too, have the ACT college entrance exams.) Other Colorado districts that serve high proportions of English language learners, including Aurora Public Schools and Adams 14 in Commerce City, also saw their ninth-grade reading and writing growth scores plummet with the switch to the PSAT.
At a statewide level, Colorado Department of Education officials noticed a similar trend. English language learners had an average growth score of just 38 in reading and writing on the PSAT and SAT. That was lower than the average growth scores for students with disabilities (43), students of color (43), and students from low-income families (42).
Zach Goldberg, a spokesman for The College Board, which makes the PSAT and SAT, said every question “is researched and pre-tested with a representative sample of students to ensure quality and prevent bias.”
In addition, Goldberg said, English language learners are allowed accommodations to make the tests more accessible to them, including test instructions written in their native language and a word-to-word bilingual glossary.
Will Morton, who oversees test administration for the state education department, said Colorado test coordinators were trained on those accommodations. But because students don’t have to ask special permission to use them, the department doesn’t track how many did, he said.
At a recent meeting, State Board of Education members took notice of the low growth scores for English language learners on the PSAT and SAT. Some wondered whether schools were doing a poor job preparing the students to take those particular tests.
State education officials don’t have answers. And they don’t have much historical context, either, because of frequent changes to the rules about which students take which tests.
In 2015, third- through 11th-graders took the PARCC standardized tests, which are aligned to the standards that dictate what students should know at each grade level. But a significant number of students in wealthy, suburban districts who saw little value in taking the PARCC tests opted out. The opt-out numbers were highest at the high school level.
So state lawmakers switched the high school tests. In 2016, 10th- and 11th-graders started taking college entrance exams instead. And in 2018, ninth-graders joined them.
Back when all students took the PARCC tests, their growth scores did not drop off in ninth grade, said Marie Huchton, the principal statistical consultant for the Colorado Department of Education. The declining trend started last year among some groups of high schoolers, including English language learners, and intensified this year, she said.
The state is investigating why. Some observers, including Derek Briggs, a University of Colorado professor and testing expert, have posited that opt-out rates might be to blame.
If more ninth-graders from wealthy, suburban districts participated in testing in 2018, and if those ninth-graders were more motivated to do well because they found the PSAT more relevant, their performance could be throwing off the year-to-year comparisons, Briggs said.
But even if that were true, he said he doesn’t think it would completely explain Denver’s big drop, for example. “It’s usually that there are multiple causes,” Briggs said.