Taking attendance

Student absences are about to have higher stakes in most states. Will cheating follow?

PHOTO: RJ Sangosti, Denver Post

Schools across the country are about to be held accountable for student attendance — attaching stakes to a measure that previously had much less significance and increasing the risk that schools will try to manipulate that data.

But it’s unclear how effectively states have prepared for that possibility, or have systems in place to accurately monitor absenteeism data at all.

“It’s human nature, when the stakes rise, to want to game the system,” said Phyllis Jordan of the Georgetown-based think tank FutureEd. She recently wrote an analysis finding that 36 states plan to use chronic absenteeism to measure schools under ESSA, the federal education law. “In that regard, I don’t think chronic absenteeism is any different than other measures, like test scores.”

Of course, one way for schools to improve their chronic absenteeism marks is to add support that helps students to show up to school. That’s exactly how experts and policymakers hope educators will respond, and because states are only using chronic absenteeism as a small portion of the accountability system, the incentives for cheating may not be strong. But past experience with evaluation systems suggests that a small number of schools will resort to unscrupulous means.

“When you’ve got high stakes on something, if there’s a way to corrupt it, some people are going to corrupt it,” said Diane Whitmore Schanzenbach, an education researcher at Northwestern who wrote a brief highlighting the advantages of using chronic absenteeism to measure schools. “The question is, how big is this incentive? How many schools are going to engage in this bad behavior?”

High stakes could lead to manipulation, but how big of a problem this will be is an open question

A 2003 study in Chicago found evidence of cheating on standardized tests in about 5 percent of elementary classrooms. More subtle gaming has also occured: research found evidence that teachers focused on topics likely to appear on the state test, at the expense of other academic standards.

The potential problem may be more acute when it comes to student absences because of the all-or-nothing way chronic absenteeism is measured.

In most states, a student is deemed chronically absent if they miss 10 percent of school days — around 18 days for those enrolled for a full school year. That means that schools might be especially tempted to mark a student present on the day of their 18th absence. (If student attendance rates are bunched right below the chronic absenteeism bar — say, many more are gone 17 rather than 18 days — that could be evidence of manipulation.)

“We need to use accountability to promote an early warning approach — not just making sure kids are one less day absent,” said Hedy Chang, the executive director of Attendance Works, a nonprofit that promotes efforts to improve school attendance.

Both Jordan and Schanzenbach noted that, because states are generally counting chronic absenteeism for only about 5 to 10 percent of school ratings, the incentives to cheat are likely to be fewer.

“It’s going be an empirical question about how big is the corruption of this — my prediction is it’s going to be reasonably small,” said Schanzenbach.

“This is why we encourage people to keep chronic absence to a relatively low percentage of the overall weighting — if it’s less than 10 percent … it’s not worth investing in trying to game it,” said Chang.

Still, attendance manipulation scandals have cropped up before: A 2016 investigation in Chicago, where student attendance rates are a part of school scores and principal evaluations, found that four high schools had systematically changed attendance records.

Others are concerned about data issues beyond obvious cheating.

“I’m worried about outright manipulation, but I’m also worried about sloppiness of reporting and inconsistencies,” said Chad Aldeman of Bellwether Education Partners, a reform-oriented consulting firm that has undertaken an extensive review of state ESSA plans. Details like how schools count partial-day absences, or what happens when a teacher forgets to take attendance, will take on new importance.

States are taking different approaches

Experts agrees that there should be some protections against manipulation of attendance data. But it’s unclear to what extent states have those safeguards in place.

Chalkbeat reached out to the 10 states that plan to use chronic absenteeism and have had their ESSA plans approved by the federal government. Nine of the state departments of education responded.

A representative for the Massachusetts Department of Education said that because the state can see changes to attendance rates, “any data manipulation would need to be done systematically and could not be done after the fact without raising flags as to why so many post-dated attendance changes were being made.”

Oregon already looks for unusual trends in attendance data, but does not conduct audits of local districts. “However, the accountability office has conducted such audits on other accountability data,” an education official said. “I would anticipate that we might do the same with attendance data submissions, should concerns arise regarding the validity of that data.”

But some states mention checks that might not catch most manipulation.

Illinois, for example, ensures that “a student can’t have more absences recorded than days enrolled in the school.”

A spokesperson from the Tennessee Department of Education noted just one kind of potential irregularity: “If the school said a student was in class all day but got in an accident or committed a crime during that window, the school could be liable.”

Some states said that they are working on this issue, but specifics have not yet been fleshed out.

Delaware “is reviewing current processes around absenteeism and chronic absenteeism.” Maine will be hiring an ESSA data coordinator “who will monitor data integrity,” but “the exact procedure and policy around this is in process of being written,” according to a spokesperson.

In Arizona, “the Department [of Education] is meeting with various stakeholders to determine ways to improve reporting of attendance and absenteeism.”

States that have already been using chronic absenteeism or attendance rates to determine funding may face less of a learning curve.

In its school performance reports, “New Jersey has included chronic absenteeism data for elementary and middle schools for years,” a spokesperson said.

Connecticut, meanwhile, has been “collecting and reporting chronic absenteeism data for many years” and has a number of checks in place, including flagging any school with large increases or decreases in chronic absenteeism, according to a spokesperson.

See all nine states’ full explanations for how they plan to protect against manipulation of absence data. Want more education news? Subscribe to Chalkbeat’s new national newsletter here.

Fact check

Why the school spending graph Betsy DeVos is sharing doesn’t mean what she says it does

PHOTO: U.S. Department of Education

U.S. Secretary of Education Betsy DeVos’s message at an event on Thursday was clear: education leaders have been throwing good money after bad policy.

To make the point, she shared a graph that appears to show that school spending has risen dramatically over the last three decades as student outcomes barely budged.

“Scores [are] continuing to muddle along, unremarkable,” she told one of her predecessors, Bill Bennett, at a Reagan Institute event. “And yet look at the spending. This is not something we’re going to spend our way out of.”

This visualization, which DeVos also shared on Twitter, has become a staple in some education policy circles. Even Bill Gates has offered a version of it. But the graph is misleading, and several recent studies have in fact shown that students gain — in higher test scores, graduation rates, and earnings as adults — when they attend schools that spend more money. And while DeVos highlighted scores in reading, gains in math have been fairly substantial over part of that period.

A spokesperson for DeVos did not immediately respond to a request for additional comment.

Here’s why it doesn’t really mean what DeVos says it does.

The biggest problem with the graphic is that it conflates correlation and causation.

Simply pointing to two trends that coincide doesn’t say anything about how one affects the other. It’s possible that if education spending hadn’t increased, test scores would have actually declined. Maybe the extra money actually stopped a dramatic deterioration in school quality, for example. There’s just no way to know based on this data.

Here’s another example: Over the past couple decades, charter schools and vouchers have expanded dramatically. By DeVos’s own logic, those would be failed reforms because they didn’t coincide with a sufficient increase in NAEP scores. But that would also be an unfair conclusion, since we don’t know what would have happened without the increase in choice programs.

The two trends are also put side-by-side in an arbitrary way.

This graph has two y axes: one based on school spending, and the other based on scale scores on the National Assessment of Educational Progress. But these two variables are very different. Should an extra $1,000 correspond with 1 point on NAEP scores? Five points?

The decision is subjective, but it determines how large the spending changes look and how small the NAEP changes look, too.

Spending is up in education, but it’s up elsewhere too.

It’s true that in inflation-adjusted dollars, school spending has risen. The economy has also grown over that same time. That matters, because if spending were to hold steady in education while increasing in other areas of the economy, it might become increasingly hard to recruit and retain teachers of equal quality.

Indeed, while education spending has risen, teacher pay compared to similar professionals has fallen. (At the same time, the number of teachers and the number of non-teaching staff in schools has increased significantly, which some have argued illustrate bad spending priorities.)

The story would also look different for math scores.

NAEP scores have been relatively flat for the past decade, but in math they increased substantially the decade beforehand. In fact, a graph could be made showing a tight correspondence between math results and spending.

That’s exactly what Northwestern University economist Kirabo Jackson did. His approach was similar to DeVos’s, with a few changes. He compared NAEP scores in math to the cumulative spending on the average student in the years leading up to the test — the idea being that scores reflect not just the last year of schooling, but several years before that.

Source: Education Next

His graphs look quite different than DeVos’s, simply by tightening the range of scores, looking at math rather than reading, and tweaking the spending measure. (Jackson said that a similar pattern exists for reading, but that it’s less pronounced.)

Recent research has generally shown that school spending really does matter.

To be clear, Jackson’s graphs are also correlational, and like DeVos’s cannot prove cause and effect.

That’s why he and others have conducted sophisticated statistical analyses to isolate the impact of resources on schools. In one recent study, Jackson and colleagues compared states that made more spending cuts in the wake of Great Recession, and found that they did worse on NAEP as a result. In other words, there is a decent case that the recent stagnation in NAEP scores is due at least in part to spending cuts.

That jibes with a larger body of research. Other recent national studies have linked more spending to better NAEP scores, higher graduation rates, better economic outcomes, and greater social mobility. Studies in specific states including California, MassachusettsOhio, and New York have also found test score gains from spending increases.

Of course, none of that proves that the additional money spent over the past few decades have been well used. It doesn’t even prove that those extra dollars have been helpful — the above studies are of specific places or time periods, and their individual approaches could be disputed, too.

There is one recent study, which DeVos often cites, showing that a federal turnaround program that involved a substantial infusion of money for struggling schools didn’t make a noticeable difference.

Still, the graph that Devos offers is not, on its own, a strong case that resources for schools don’t make a difference. Available evidence suggests the opposite.

“If reduced spending contributed to the problem, it seems reasonable to conclude that increased spending may alleviate it,” said Jackson, the Northwestern researcher. “This is not to say that increased spending is some silver bullet, but rather that it should probably be part of a comprehensive set of policies to improve U.S schools.”

after douglas

Betsy DeVos avoids questions on discrimination as school safety debates reach Congress

Education Secretary Betsy DeVos prepares to testify at a House Appropriations Labor, Health and Human Services, Education and Related Agencies Subcommittee hearing in Rayburn Building on the department's FY2019 budget on March 20, 2018. (Photo By Tom Williams/CQ Roll Call)

U.S. Education Secretary Betsy DeVos fielded some hostile questions on school safety and racial discrimination as she defended the Trump administration’s budget proposal in a House committee hearing on Tuesday.

The tone for the hearing was set early by ranking Democrat Rep. Rosa DeLauro, who called aspects DeVos’s prepared remarks “misleading and cynical” before the secretary had spoken. Even the Republican subcommittee chair, Rep. Tom Cole, expressed some skepticism, saying he was “concerned about the administration continuing to request cuts that Congress has rejected.”

During nearly two hours of questioning, DeVos stuck to familiar talking points and largely side-stepped the tougher queries from Democrats, even as many interrupted her.

For instance, when Rep. Barbara Lee, a Democrat from California, complained about proposed spending cuts and asked, “Isn’t it your job to ensure that schools aren’t executing harsher punishments for the same behavior because [students] are black or brown?” DeVos responded by saying that students of color would benefit from expanded school choice programs.

Lee responded: “You still haven’t talked about the issue in public schools as it relates to black and brown students and the high disparity rates as it relates to suspensions and expulsions. Is race a factor? Do you believe that or not?” (Recent research in Louisiana found that black students receive longer suspensions than white students involved in the same fights, though the difference was very small.)

Again, DeVos did not reply directly.

“There is no place for discrimination and there is no tolerance for discrimination, and we will continue to uphold that,” she said. “I’m very proud of the record of the Office of Civil Rights in continuing to address issues that arise to that level.”

Lee responded that the administration has proposed cuts to that office; DeVos said the reduction was modest — less than 1 percent — and that “they are able to do more with less.”

The specific policy decision that DeVos faces is the future of a directive issued in 2014 by the Obama administration designed to push school districts to reduce racial disparities in suspensions and expulsions. Conservatives and some teachers have pushed DeVos to rescind this guidance, while civil rights groups have said it is crucial for ensuring black and Hispanic students are not discriminated against.

That was a focus of another hearing in the House on Tuesday precipitated by the shooting last month at Marjory Stoneman Douglas High School in Parkland, Florida.

Rep. Jim Sensenbrenner, a Wisconsin Republican, falsely claimed in his opening statement that Broward County Public Schools rewrote its discipline policy based on the federal guidance — an idea that has percolated through conservative media for weeks and been promoted by other lawmakers, including Florida Sen. Marco Rubio and Utah Sen. Mike Lee. In fact, the Broward County rules were put into place in 2013, before the Obama administration guidance was issued.

The Manhattan Institute’s Max Eden, a leading critic of Obama administration’s guidance, acknowledged in his own testimony that the Broward policy predated these rules. But he suggested that policies like Broward’s and the Obama administration’s guidance have made schools less safe.

“Faced with pressure to get the numbers down, the easiest path is to simply not address, or to not record, troubling, even violent, behavior,” he said.

Kristen Harper, a director with research group Child Trends and a former Obama administration official, disagreed. “To put it simply, neither the purpose nor the letter of the federal school discipline guidance restrict the authority of school personnel to remove a child who is threatening student safety,” she said.

There is little, if any, specific evidence linking Broward County’s policies to how Stoneman Douglas shooter Nicholas Cruz was dealt with. There’s also limited evidence about whether reducing suspensions makes schools less safe.

Eden pointed to a study in Philadelphia showing that the city’s ban on suspensions coincided with a drop in test scores and attendance in some schools. But those results are difficult to interpret because the prohibition was not fully implemented in many schools. He also cited surveys of teachers expressing concerns about safety in the classroom including in Oklahoma CityFresno, California; and Buffalo, New York.

On the other hand, a recent study found that after Chicago modestly reduced suspensions for the most severe behaviors, student test scores and attendance jumped without any decline in how safe students felt.

DeVos is now set to consider the repeal of those policies on the Trump administration’s school safety committee, which she will chair.

On Tuesday, DeVos said the committee’s first meeting would take place “within the next few weeks.” Its members will be four Cabinet secretaries: DeVos herself, Attorney General Jeff Sessions, Secretary of Health and Human Services Alex Azar, and Secretary of Homeland Security Kirstjen Nielsen.

Correction: An earlier version of this story said that Rep. Barbara Lee was from Texas; in fact, she represents a California congressional district.