On the first day of her first college math class, JaniQua Guiste chose a seat near the front of a computer lab at Rutgers University-Newark. Right at 10 a.m., the instructor looked up from her computer.
“Good morning, students,” she said that sunny morning this September. “How are you today?”
Guiste hadn’t been surprised to test into Math 104, or “Intermediate Algebra,” a remedial class that begins with whole numbers and fractions before moving into linear equations and rational expressions. It was material she might have mastered in 11th-grade Algebra 2 at her Newark high school had the teacher not quit the second week, leaving the class with a rotating cast of substitutes.
“In math, I knew I wasn’t going to be prepared,” said Guiste, a native of the U.S. Virgin Islands who moved to Newark in 2017 after Hurricane Irma leveled her family’s home.
Now her math instructor was telling students to log onto their computers and register for ALEKS, an online program that would play the role of co-teacher. A few minutes later, a student in her 30s asked if they would use a workbook along with ALEKS.
“We don’t have a workbook,” the instructor replied. “Everything is online.”
That comment would shape the semester for Guiste and thousands of other students in remedial courses across the country, where more than 40% of public colleges have turned to online and other self-paced programs to help their most struggling students get up to speed in math.
Unlike lecture courses, online programs allow students to plug away at their own speed on problems matched to their individual learning needs — a format that works well for some students. But there are also pitfalls to that approach.
Students who arrive with the biggest gaps in their learning can easily fall behind in computer-based courses — especially students “who don’t have great study skills and aren’t super self-directed,” said Maggie Fay, a researcher who has studied such classes. And even in supervised courses like Guiste’s, the instructors can struggle to catch students who veer off track.
“The theory of action on online remediation sounds very promising,” said Fay, who is a research associate at the Community College Research Center at Teachers College, Columbia University. But most of the independent research has “not shown super strong outcomes.”
Turning to tech
The rise of computer-based remediation is one response to a longstanding crisis in higher education: Many of the students who enter college furthest behind never catch up.
This year, Chalkbeat reporters in Newark and Detroit are examining whether students from struggling schools are prepared for college — and whether colleges are prepared for them. Catch up on the Ready or Not series here.
Those students are typically placed in remedial classes, which cover material often taught in high school or earlier. Remediation allows community colleges and less-selective four-year institutions to serve under-prepared students who might otherwise be shut out of higher education.
But what was intended as an on-ramp to college often becomes a parking lot: About half of remedial course-takers at community colleges and roughly 40% at four-year institutions never complete the courses, according to a 2016 federal study, which found that black, Hispanic, and low-income students were overrepresented in remediation. Stymied by the courses, many students leave college with debt and dashed hopes instead of degrees.
Those dismal outcomes have sparked a national reform movement, with colleges and entire states taking steps to send fewer students to remediation or allow them to skip it entirely. At the same time, many have overhauled the way they deliver remediation to students who need it — in many cases, with the help of technology.
Today, more than 40% of four-year public colleges now offer self-paced remedial math classes that often rely heavily on computers, according to a recent report by the Center for the Analysis of Postsecondary Readiness. A new effort backed by the Bill & Melinda Gates Foundation aims to bring the “adaptive learning” technology that powers many of those classes to 200 more colleges in the coming years.
Adaptive learning programs like ALEKS make an alluring promise: college, but “personalized.” Instead of survey courses in anonymous lecture halls, students get customized online lessons with material adapted to their needs.
“When I’m teaching a lecture, I’m teaching you something whether you already know it or not,” said Robert Puhak, who directs college algebra and basic math at Rutgers-Newark and oversees the ALEKS classes. “This is just going after the things you don’t know.”
Theory of action
In Guiste, Rutgers-Newark had found an ideal student for its computer-based math class.
She prefers working online over copying notes off a whiteboard. She isn’t afraid to flag down the instructor when a problem baffles her. She’s meticulous about completing her assignments, which she lists on a weekly calendar in her dorm room. And she likes learning from ALEKS.
“It breaks down every step,” she said. “It has a problem, then it has an answer to the problem — like a whole explanation.”
Adaptive programs allow students to skip topics they already know and focus on those they don’t. At Rutgers-Newark, students work through practice problems individually as an instructor stands by to answer questions and teach brief lessons when needed. To advance to a new topic, students must answer several questions correctly.
"The whole purpose is that it's going to teach you things you don't know."
To do all that, ALEKS must first determine what students know. So during that first session of Math 104 in September, the students took an online “Knowledge Check” that assessed their grasp of the 341 topics covered in the first portion of the class. ALEKS returned the results immediately: One student in the back row who raced through the quiz understood 151 of the topics; two seats over, his classmate knew only 69.
The instructor told the class not to stress: “The whole purpose is that it’s going to teach you things you don’t know.”
A commuter campus situated in the heart of the city, Rutgers-Newark proudly educates a wide array of students — including about 15% who hail from Newark. That commitment to diversity also means that many students arrive needing some remediation.
That has not always gone well. A 2003 report by external evaluators said Rutgers-Newark had “a very weak developmental program.” (Some researchers and college administrators refer to “developmental” rather than remedial education, which they believe has negative connotations.)
Since then, Rutgers-Newark has ramped up student supports such as tutoring, advising, and financial aid, while condensing remedial math into a single semester (a move shown to boost completion rates) and hiring full-time faculty members to teach remedial classes instead of part-time instructors. And in 2015, the campus began using ALEKS for remedial math.
The promise of personalized learning has captivated many colleges. In 2018, nearly 2 million college students used ALEKS, one of several online programs on the market.
Yet the limited research on computer-based remediation has found mixed outcomes for “hybrid” courses like those at Rutgers-Newark, where students use online programs in computer labs under the guidance of instructors.
In Tennessee, students at four-year colleges who took remedial math on computers were more likely to pass the course than students in lecture classes but less likely to pass their first college-level math class, according to a new study. And in Kentucky, community college students who took computer-based remediation were less likely to pass the course and less likely to earn degrees than students in traditional classes, according to a recent peer-reviewed study.
“We have seen a lot of colleges move quickly towards adopting these computer-based instructional models” for remediation, said Angela Boatman, a higher-education professor at Boston College who conducted the Tennessee study. The limited rigorous research to date, she added, raises the possibility that “this is not the solution that colleges had hoped it would be.”
Still, some colleges have seen improvements after turning to tech. McGraw-Hill Education, the publisher that owns ALEKS, points to several case studies suggesting positive outcomes at institutions that use its program.
For instance, Arizona State University and a Milwaukee community college both found that students who used ALEKS were more likely to pass basic math courses than students in traditional classes. The studies were conducted by people at those institutions or researchers connected to McGraw-Hill; neither looked at students’ later outcomes.
“They’re seeing these improvements they just haven’t been able to see in the classic models,” said Scott Virkler, chief product and operations officer at McGraw-Hill Higher Education. ALEKS, he added, “works exceptionally well for remediation.”
‘They were really lost’
Essex County College had hoped online remediation would deliver the miracle it needed.
A community college adjacent to Rutgers-Newark, Essex is a top destination for Newark residents seeking better lives — some fresh out of high school, others returning to the classroom after long absences. Nearly 43% of graduates from Newark’s non-selective high schools enroll at Essex, compared to about 3% at Rutgers-Newark, according to a recent study.
Like most two-year colleges, Essex accepts anyone who applies. The vast majority of incoming students must take at least one remedial class, but few make it out. In math, less than 7% of students who start in the lowest level of remediation ultimately pass a college-level math course.
“Developmental math is a swamp where a lot of people never escape,” said Douglas Walcerz, a former vice president at Essex.
Walcerz was convinced that the format of remedial math was fatally flawed. No instructor could give a lesson that met the needs of students whose math skills ranged from an 11th-grade to a fourth-grade level, he said.
So in 2013, Walcerz brought in ALEKS. Using federal funds and a $100,000 grant from the Gates Foundation, Essex piloted a new remedial math course where students met in computer labs three days a week to work online with guidance from instructors. Two days a week, they took a mandatory study-skills class where they set weekly goals for topics to complete in ALEKS.
When students took a diagnostic exam at the beginning of the course that fall, the results confirmed Walcerz’s suspicion about the huge variation among students: The least prepared knew less than 5% of the course topics, while the most prepared knew more than 70%.
"Developmental math is a swamp where a lot of people never escape."
ALEKS enabled a small number of students to zip ahead, finishing the class early or completing two courses in one semester. But others were left in the dust. Pass rates plummeted in the classes that used ALEKS the first year, which Walcerz later called “disastrous.” However, the ALEKS students who did pass were slightly more likely to make it through college-level math than their peers who took regular remedial classes, said Walcerz, now a dean at Brazosport College in Texas.
The results were also disappointing at other colleges and universities that received Gates funding to experiment with adaptive-learning programs. A 2016 Gates-commissioned study that reviewed data from 14 institutions found the technology did not generally improve course completion, and only led to modest learning gains at three institutions. “The ultimate goal — better student outcomes at lower cost — remains elusive,” the report concluded. (Chalkbeat receives some funding from Gates.)
In 2016, Essex abandoned ALEKS.
Brooke Orosz, an assistant professor of math and physics at Essex, said it had been hard for instructors to keep tabs on a room full of students each practicing different topics online. And without group work, students had few opportunities to learn from their peers. That placed a heavy burden on students to learn the material themselves using ALEKS and to seek help when they got stuck.
“It seemed like a third of the class was doing great with it and the other two-thirds didn’t really know what to do,” she said. “They were really lost.”
Today, the college is planning a number of research-backed reforms that don’t rely on technology. Those include using multiple measures instead of a single test to determine if students need remediation, embedding tutors in some remedial classes, and piloting a college-level “co-requisite” course covering statistics and related algebra so students can skip a semester of remedial algebra.
“We really believe that if we could help students get through developmental math,” said Jill Stein, the dean who oversees math and other subjects, “it would have a huge impact on retention and graduation.”
The human component
At first, Math 104 was a breeze for Guiste.
She had attended a summer “boot camp” at Rutgers-Newark that gave low-income, less academically prepared students a crash course in surviving college. During the program, she took a math class where she learned concepts also covered in the first weeks of Math 104.
But as the fall semester wore on, the material grew more challenging and the pace more grueling. The class included weekly quizzes on top of the ALEKS practice problems. To keep up, Guiste tried to master six topics per day — a goal that required correctly answering dozens of practice problems, often late at night in her dorm room.
In class, she mostly learned alone. Unlike during the summer, she and her classmates didn’t get to work through problems together at the whiteboard. The instructor gave brief lessons, but Guiste often found them confusing — especially when the topics differed from the ones in Guiste’s customized ALEKS course.
She could usually figure out how to solve problems by poring over the examples that ALEKS gave. But because cost constraints had kept Rutgers-Newark from purchasing a textbook or instructional videos to supplement ALEKS, Guiste often turned to Google and YouTube when a concept stumped her.
“What’s the point of being in the class with a teacher if you’re going to teach yourself with ALEKS?” she said one dreary afternoon in October.
In class one morning that month, it was clear that Guiste wasn’t the only student teaching herself.
"What's the point of being in the class with a teacher if you're going to teach yourself with ALEKS?"
The students were scheduled to take a quiz on linear equations — their seventh quiz in six weeks. The following week would bring more quizzes and an exam, plus all the ALEKS questions for the first portion of class would be due. Yet when class started at 10 a.m., only seven of 23 students were present in the computer lab.
The instructor wrote a practice problem on the board — ¾ x – ⅔ = ⅙ — then briskly narrated the steps to solve it. But hardly anyone watched. Instead, students stared intently at their computers, some with their hoods up or headphones on to drown out distractions. They worked on a practice quiz or problems in ALEKS. A couple students raised their hands to request help with their ALEKS problems. Sitting in the front row, Guiste asked when they could start the quiz.
Around 10:30, it began. One student who wasn’t in the room tried accessing the quiz remotely.
“Someone’s trying to trick me,” the instructor said. (The instructor allowed Chalkbeat to observe her class multiple times on the condition that she not be named.)
About 30 minutes later, most students had completed the quiz’s eight problems. Circulating through the room, the instructor saw that one young man had answered three problems incorrectly.
“What happened? We just did this on the board. You weren’t listening?” she asked him. “I came in late,” said the student, Jake Lakatos.
That quiz was an exception for Lakatos. He had earned perfect scores on most of the other quizzes, which covered material he recalled learning at his high school in a middle-class township south of Newark. For him, the format of the class was ideal.
“I listen in if there is something I need help with,” he said about the instructor’s lessons. “But most of the time, I’m just kind of tuned out and going ahead on the computer.”
But that approach fell short for students such as Jillian Rock, who asked about workbooks on the first day of class. A writer and visual artist, she had taken classes at other colleges before enrolling at Rutgers-Newark and placing into remedial math.
As Rock progressed slowly through ALEKS, relearning concepts she hadn’t used in years, she bumped up against the faster pace of the course. One day, she had to pause her online work on fractions to take a class quiz on geometry. Meanwhile, she missed learning from peers and her professor as she had in previous math classes.
“There’s no human component to this,” she said, “and I think that’s where I struggle.”
Eventually she dropped the course. She planned to retake it later — in a lecture format.
Teaching with ALEKS
Instructors are expected to play a pivotal role in hybrid classes like Guiste’s, harnessing the software to track students’ progress and swoop in strategically.
“ALEKS does not replace the teacher,” said Virkler of McGraw-Hill Higher Education. “The teacher is the most important thing — that doesn’t change.”
"ALEKS does not replace the teacher"
Programs like ALEKS can provide instructors with a powerful tool. With a few clicks, they can see how long a student was logged into the program, what problems they got wrong, and how many attempts it took to get the right answers. Armed with that data, instructors know “where to spend your time and energy and where each student is having their unique set of challenges,” Virkler said.
But successfully making use of the data requires time and training that are often in short supply.
Jonathan Guhl-Miller, a part-time instructor at Rutgers-Newark who teaches Math 104 and other classes, said ALEKS’s trove of data is only useful if instructors can sort through it all.
“It does require an enormous amount of paying attention to the extra tools” to monitor each person’s progress in a class of 20 or more students, he said.
Data analysis is just one of many skills required to teach alongside a computer. Instructors must keep everyone on track even as students work at their own pace. And they must intervene seamlessly, pivoting from private coaching one minute to a whole-class lesson the next.
“It’s a new type of challenge,” said Puhak, the Rutgers-Newark math director who oversees ALEKS classes. “You have to be savvy as to who knows what — and more so — where does each individual need help and then how can you best do that.”
Puhak said he hopes to offer instructors more training in teaching strategies, which college faculty across the country often never receive. He’s inspired by seeing what computer-centered remedial classes look like when led by expert educators — like Wahab Ashraf.
Ashraf, who graduated from Rutgers-Newark and also teaches honors biology at a Newark high school, has taught Math 104 for several years. During one of his evening classes last month, he seemed to fuse the interactive, supportive structure of high school with the more independent approach of college.
“Hello, hello, hello,” he said as students took their seats in the computer lab. “Hello, professor,” several replied.
Like Guiste’s class the previous month, Ashraf’s students were preparing for a quiz the next day. But Ashraf’s review looked completely different. He asked students to pair up and solve a series of practice problems. While they worked, he scanned their calculations and pointed out mistakes or asked a student from one pair to explain a concept to a different duo.
In the third row, a young man walked his partner through the steps for using an equation to find the area of a rectangle. She followed his guidance and got the right answer.
Only after Ashraf called students to the board to show how they solved the problems did he allow independent work on computers.
“ALEKS just gives them notes — it maps out the steps, but sometimes they can’t see how ALEKS goes from one step to the next,” Ashraf said, adding that is why group work and the instructor’s lessons are invaluable. “It’s a huge benefit to just have someone there to help guide their learning.”
Making classes like Ashraf’s into the norm could be a challenge. Guhl-Miller said that while he does give some lectures in Math 104, that is complicated by the fact that students are “working on different topics at different times” in ALEKS. He also offers extra assistance to anyone who needs it during or after class, he added. But ultimately, he said, students must take responsibility for their own learning.
“This is the college level; we are not looking over their shoulder all the time,” he said. “If they don’t realize they need help and they won’t accept my help, then they’re going to struggle and fail.”
Preparing for what’s next
By one measure, Rutgers-Newark’s remedial math program is working for most students. On average 75% of students pass Math 104 and Math 105, which enrolls students who score slightly higher on the placement test but still require remediation.
But experts say the real test of remediation is whether it prepares students to succeed in college-level classes — and on that front, Rutgers-Newark has a mixed record. In the 2017-18 school year, up to 90% of students who completed remedial math passed Math 106, a liberal arts course that does not involve algebra, according to administrators. But just 50-57% of remedial math completers made it through Math 109, college-level algebra for business and science.
“We have to keep evaluating what we’re doing and adapting our techniques, our models, to better serve students,” Puhak said.
Other colleges that rely on computer-based remediation should do the same, said Nikki Edgecombe, principal investigator for the Center for the Analysis of Postsecondary Readiness.
Despite evidence of “weak outcomes” for some students in online courses, colleges, philanthropies, and tech companies are unlikely to abandon an approach they’ve invested heavily in, she added. So instead, researchers and practitioners should find improvements.
“We need to figure that out,” Edgecombe said. “Because otherwise, it’s a space where we’re going to exacerbate inequity.”
For her part, Guiste is determined to finish Math 104 and tackle the more advanced algebra she’ll need to earn a degree and, one day, become a dentist.
"We need to figure that out. Because otherwise, it's a space where we're going to exacerbate inequity."
So around 4 p.m. one afternoon in early December, with the final exam less than two weeks away, she found a quiet lounge in her dorm and pulled up ALEKS on her laptop. She realized that between class and her campus job as an office assistant, she had never eaten lunch. Yet she kept working.
She took a practice quiz in ALEKS, then read and reread its explanation of a problem she got wrong on graphing a line. It still didn’t make sense, so she resolved to ask a friend in higher-level math for help.
Thinking back that afternoon on her long slog through Math 104, she decided the hard work had been worth it: “I feel like all of it is going to prepare us for what’s next.” But first, she needed to pass.
“I just really hope and pray,” she said.
This project was produced with support from the Education Writers Association Reporting Fellowship program.