evaluating evaluation

How bias happens: teaching struggling students can affect observation scores, study finds

PHOTO: Thomas Barwick | Getty Images

A teacher is observed in her first period class and gets a low rating; in her second period class she gets higher marks. She’s teaching the same material in the same way — why are the results different?

A new study points to an answer: the types of students teachers instruct may influence how administrators evaluate their performance. More low-achieving, black, Hispanic, and male students lead to lower scores. And that phenomenon hurts some teachers more than others: Black teachers are more likely to teach low-performing students and students of color.

Separately, the study finds that male teachers tend to get lower ratings, though it’s not clear if that’s due to differences in actual performance or bias.

The results suggest that evaluations are one reason teachers may be deterred from working in classrooms where students lag farthest behind.

The study, conducted by Shanyce Campbell at the University of California, Irvine, analyzed teacher ratings compiled by the Measures of Effective Teaching Project, an effort funded by the Bill and Melinda Gates Foundation. (Gates is also a supporter of Chalkbeat.)

The paper finds that for every 25 percent increase in black or Hispanic students taught, there was a dip in teacher’s rating, similar to the difference in performance between a first and second-year teacher. (Having more low-performing or male students had a slightly smaller effect.)

That’s troubling, Campbell said, because it means that teachers of color — who often most frequently work with students of color — may not be getting a fair shot.

“If evaluations are inequitable, then this further pushes them out,” Campbell said.

The findings are consistent with previous research that shows how classroom evaluations can be biased by the students teachers serve.

Cory Cain, an assistant principal and teacher at the Urban Prep charter network in Chicago, said he and his school often grapple with questions of bias when trying to evaluating teachers fairly. His school serves only boys and its students are predominantly black.

“We’re very clear that everyone is susceptible to bias. It doesn’t matter what’s your race or ethnicity,” he said.

While Cain is black, it doesn’t mean that he doesn’t see how black boys are portrayed in the media, he said. And also he knows that teachers are often nervous they will do poorly on their evaluations if students are misbehaving or are struggling with the content on a given day, knowing that it can be difficult for observers to fully assess their teaching in short sessions.

The study, co-authored by Matthew Ronfeldt of the University of Michigan, can’t show why evaluation scores are skewed, but one potential explanation is that classrooms appear higher-functioning when students are higher-achieving, even if that’s not because of the teacher. In that sense, the results might not be due to bias itself, but to conflating student success with teacher performance.

Campbell said she hopes her findings will add nuance to the debate over the best ways to judge teachers.

One idea that the study floats to address the issue is an adjustment of evaluation scores based on the composition of the classroom, similar to what is done for value-added scores, though the idea has received some pushback, Campbell said.

“I’m not saying we throw them both out,” Campbell said of classroom observations and value-added scores. “I’m saying we need to be mindful.”

negotiations

Aurora school board reverses course, accepts finding that district should have negotiated bonuses with union

Students in a math class at Aurora Central High School in April 2017. (Photo by Yesenia Robles, Chalkbeat)

Following weeks of criticism, the Aurora school board on Tuesday reversed course and accepted an arbitrator’s finding that a pilot bonus system violated the district’s agreement with the teachers union.

The Aurora school district rolled out an experiment last year to offer bonuses to some teachers and other staff in hard-to-fill positions, such as psychologists, nurses and speech language pathologists.

The teachers union argued that the plan should have been negotiated first. An arbitrator agreed and issued a report recommending that the pilot program stop immediately and that the district negotiate any future offerings. The union and school board are set to start negotiations next month about how to change teacher pay, using new money voters approved in November.

When school board members first considered the arbitrator’s report last month, they declined to accept the findings, which were not binding. That raised concerns for union members that the district might implement bonuses again without first negotiating them.

Tuesday’s new resolution, approved on a 5-1 vote, accepted the full arbitrator’s report and its recommendations. Board member Monica Colbert voted against the motion, and board member Kevin Cox was absent.

Back in January 2018, school board members approved a budget amendment that included $1.8 million to create the pilot for incentivizing hard-to-fill positions. On Tuesday, board member Cathy Wildman said she thought through the budget vote, the school board may have allowed the district to create that incentive program, even though the board now accepts the finding that they should have worked with union before trying this experiment.

“It was a board decision at that time to spend that amount on hard-to-fill positions,” Wildman said.

Board president Marques Ivey said he was not initially convinced by the arbitrator’s position, but said that he later read more and felt he could change his vote based on having more information.

Last month, the Aurora school board discussed the report with its attorney in a closed-door executive session. When the board met in public afterward, it chose not to uphold the entire report, saying that the board could not “come to an agreement.” Instead board members voted on a resolution that asked the school district to negotiate any future “long-term” incentive programs.

Union president Bruce Wilcox called the resolution “poorly worded” and slammed the board for not having the discussion in public, calling it a “backroom deal.” Several other teachers also spoke to the board earlier this month, reminding the newest board members’ of their campaign promises to increase transparency.

Board members responded by saying that they did not hold an official vote; rather the board was only deciding how to proceed in public. Colorado law prohibits schools boards from taking positions, or votes, in private.

The board on Tuesday also pushed the district to provide more detailed information about the results of the pilot and survey results that tried to quantify how it affected teachers deciding to work in Aurora.



story slam

The state of teacher pay in Indiana: Hear true stories told by local educators

It’s time to hear directly from educators about the state of teacher pay in Indiana.

Join us for another Teacher Story Slam, co-hosted by the Eiteljorg Museum of American Indians and Western Art, Chalkbeat Indiana, and Teachers Lounge Indy. Teacher salaries are the hot topic in education these days, in Indiana and across the country. Hear from Indianapolis-area teachers who will tell true stories about how they live on a teacher’s salary.

Over the past two years, Chalkbeat has brought readers personal stories from the teachers, students, and leaders of Indianapolis through our occasional series, What’s Your Education Story? Some of our favorites were told live during teacher story slams hosted by Teachers Lounge Indy.

Those stories include one teacher’s brutally honest reflection on the first year of teaching and another teacher’s uphill battle to win the trust of her most skeptical student.

Event details

The event will be held from 6-8 p.m. on Friday, March 15, at Clowes Court at the Eiteljorg, 500 W Washington St. in Indianapolis. It is free and open to the public — please RSVP.

More in What's Your Education Story?