evaluating evaluation

More work, worse relationships, and better feedback: How teacher evaluation has changed the job of the principal

PHOTO: Laura Faith Kebede/Chalkbeat
Principal Melody Smith discusses how students at A.B. Hill Elementary grew significantly in test scores.

Teacher evaluation overhauls were supposed to reshape the teaching profession. New research suggests they may have had an even greater impact on what it means to be a school principal.

As policy makers overhauled teacher rating systems in the last decade, principals began spending much more of their time watching teachers in action and talking to them about how to improve. But the shift also overwhelmed them with work, stopped them from fulfilling other responsibilities in their schools, and weakened their relationships with teachers.

Those are some of the takeaways of a study based on interviews with dozens of principals in six districts that revamped how teachers were evaluated in the last decade. Typically, the new evaluation systems — often put in place at the urging of the federal government and influential philanthropies, particularly the Gates Foundation — incorporated student performance for the first time but were driven mostly by teachers’ scores on rubrics that attempt to spell out good teaching practices.

While the systems’ adoption often precipitated significant pushback and in some cases have been rolled back, they remain in force in many places — and their long-term consequences are only now becoming clear.

The study, published last week in The Elementary School Journal, focuses on six large districts that embarked on dramatic evaluation changes: Baltimore, Denver, Hillsborough County (Florida), Houston, Memphis, and Metro Nashville. The researchers interviewed 10 principals in each district for about an hour during the 2012-13 school years, selecting a group at random while also ensuring a combination in different grade and performance levels.

One finding stood out. “The role of administrator is drastically changing,” one principal said. “You cannot be just a manager of a building. … You have to be an instructional leader first.”

The study examines only a handful of the many districts that reworked evaluations in the last decade, so it’s unclear how widely the findings apply. But the researchers detected themes across the six locations, which all had somewhat different evaluation systems and strategies for rolling them out. Here’s what they heard.

The new evaluations take way more time than principals feel they have.

The most consistent concern shared with researchers centered on the time it takes to complete the new evaluations. The responsibilities didn’t just include observing teachers more frequently, but also conducting pre- and post-observation conferences, completing additional paperwork and data entry, and using final ratings to make human resources decisions.

“I mean, honest to God. I just can’t do it by myself,” one principal told the researchers. “If I’m running my school, something’s going to lose out, either academics or your school because you’re just one person.”

“We need to [have] a preconference, an observation, and a post conference, you’re talking about, even though a preconference and postconference may only take 20 to 30 minutes and then an observation, you’re looking at two hours,” another principal said.

“It took us like an hour and a half to input the evaluations per teacher, and I have 82 people. So it was like a nightmare, literally,” said another.

In New York City, where the recently proposed teachers union contract reduces the number of required observation from four to two for most teachers, principals say the blanket requirements of the new system impeded their ability to make good choices about how they spend their time.

“We’re better off spending more time with the teachers who need our help than with the teachers who are doing a bang-up job,” the president of the union that represents school administrators told Chalkbeat. “To say time is at a premium for our members is an understatement.”

Principals are spending more time with teachers — and many say that’s a good thing.

Although the use of test scores has driven much of recent public debates about evaluations, the principals interviewed in the study often focused on the role of observations. That’s in line with past research suggesting principals trust that measure more than test scores; it’s also not surprising since principals conducted observations themselves, and those scores accounted for the lion’s share of a teacher’s overall rating.

While some frameworks for evaluating instruction have gotten mixed reviews, principals in the study said they found them helpful. Having an “objective” rubric, the principals told the researchers, allowed them to both give specific feedback to teachers who were improving and to build evidence to dismiss teachers who persistently struggled.

“For me, as a leader, it has given me a tool that I can use to help teachers grow, more than I’ve ever been able to do in the past, ever, because it’s a lot more specific than it ever has been in the past,” one principal said.

“The … rubric is the single most solid evaluative tool that I’ve used in my career,” said another.

“The conversations and the reflection … have gone from, ‘Oh, I thought it went great,’ to a really robust conversation.”  said a third principal.

Whether observations are really as “objective” as principals seemed to believe is not clear. Other research has shown that classroom observations are consistently biased against teachers who serve more students of color and students who start the year with lower test scores.

Principals’ relationships with teachers worsened.

Though there has been modest uptick in teachers who received low ratings, the vast majority of teachers across the country were still rated effective under the new systems.

Still, principals said the new evaluations damaged their relationships with teachers, who weren’t always sold that the new approaches were fair. They cited this dynamic as a major downside of new evaluation systems.

Some principals framed the tension as a lack of understanding on teachers’ part. “It’s very difficult for some teachers to understand that you really can quantify what they’re doing in the classroom, and many of them don’t want you quantifying it,” one said.

Others said the culture in the school had changed as teachers became more fearful of high-stakes evaluation, and thus less open with their principal.

“I feel like every time I walk in, a teacher’s like automatically on pins and needles thinking I’m there in an evaluative capacity,” a principal told researchers. “I just want to go in and see what the kids are doing.”

Principals ended up being less visible in their schools.

One surprising finding is that even as the new evaluation systems required principals to spend lots of time in classrooms, they said they ended up being less visible to educators and students.

“I’m in more classrooms, but I’m not walking through lots of classes. I’m mainly only in four classes or three classes a day,” one principal explained.

“I don’t do lunch duty as much,” another said. “I loved lunch duty because it’s a time to interact with the kids where you can just communicate with them.”

Trisha Arnold, a New York City teacher who helped negotiate the contract terms that reduce required observations there, told Chalkbeat earlier this month that she had experienced this phenomenon in her school.

“Kids aren’t sure who the principal is,” she said, “because they’re bogged down with paperwork.”

What to do with the research is unclear.

How do the benefits and costs of new systems weigh out? That remains a frequently debated question among policymakers and researchers. A recent analysis focusing on the Gates Foundation’s evaluation reforms in a number of districts yielded disappointing results, with little if any gains in achievement. (Two of those districts were the same ones featured in the latest study, which was also funded by the Gates Foundation; additionally, Gates is a supporter of Chalkbeat.) Another study found that evaluations systems likely deterred prospective teachers from entering the profession.

Other research is more upbeat, linking evaluation to test score gains and teacher improvement.  Studies of evaluation efforts in Cincinnati, Chicago, Washington D.C., and Minnesota have all found benefits, as well as national research on performance pay and feedback connected to classroom observations.

The latest research suggests that finding ways to lighten principals’ workloads might be worthwhile. That’s already happened to a degree, with some principals saying they placed more responsibility on assistant principals or other administrators. A separate Washington State study found a hiring spree of assistant principals soon after evaluation systems were put into place. That might be a smart move to help principals — but it also costs money, which could have been spent elsewhere, say, increasing teacher pay or reducing class size.

The new research also underscores how significant the choice of observation rubric is for districts and schools that adopt new evaluation systems. “Policy makers may also want to consider the enormous weight that is being placed on instructional rubrics as a means of guiding principals’ understanding of teacher performance,” the authors say.

Perhaps most importantly, the authors say, there should be more attention on principals’ role in enacting the raft of procedures connected to new evaluations.

“Without a careful examination of how to address principals’ concerns about lack of time,” they conclude, “the risk of principal fatigue, frustration, and eventual turnover remains substantial.”

ready for prime time

Four ways Amazon’s arrival in New York City could impact public schools

PHOTO: Reema Amin/Chalkbeat
John Schoettler, Amazon's vice president of global real estate and facilities, (left) sits with Gov. Andrew Cuomo and Mayor Bill de Blasio at a press conference about Amazon's announcement to open part of its new headquarters in Long Island City.

After months of speculation, Amazon announced Tuesday that it picked Long Island City for one of its two new headquarters.

Details about the new Queens hub are still emerging, and some of the particulars are already raising eyebrows — including billions in incentives Amazon was offered to locate here. The deal, which officials claim will create as many as 40,000 jobs over 10 to 15 years, will undoubtedly affect New York City’s public school system.

The formal agreement between Amazon and New York City lays out several direct ways that the deal could impact city schools. The company agreed to house a 600-seat intermediate school on or near its Long Island City campus, replacing a school that had already been planned in a residential building nearby. Amazon also plans to offer “career exploration activities” and internship opportunities to high school students. And there is a proposal to relocate some Department of Education offices in Long Island City to make way for the tech giant.

If Amazon’s impact on Seattle, its primary headquarters, is any guide, there could be reverberations felt in New York City classrooms, especially those districts in or adjacent to Long Island City. Still, given New York City’s size and economy, Amazon’s arrival may not create the same sweeping changes — and officials are already trying to reassure New Yorkers.

“The city and state are working closely together to make sure Amazon’s expansion is planned smartly, and to ensure this fast growing neighborhood has the transportation, schools, and infrastructure it needs,” de Blasio said in an Amazon blog post announcing the move.

Here are four potential issues to look out for.

Overcrowded schools

Amazon has pledged to donate space for a new middle school — space that parents say is desperately needed. De Blasio said Tuesday that the school will replace another that had been proposed for the area. “There is no loss of school seats,” he said.

But Meghan Cirrito, a member of the Gantry Parent Association, an education advocacy group in Long Island City, is skeptical that the school will ease the crunch for classrooms. Queens parents have long fought for more school space in the borough. In the Long Island City neighborhood, schools that serve grades K-8 are already at 102 percent capacity.

“It will absolutely not relieve the overcrowding. They will keep up with their own development,” she said. “We’re already behind school seats.”

Deborah Alexander, the co-president of Community Education Council 30, which includes Long Island City, echoed that the school plan feels like “a pittance.”

“We’re still playing catch up for the city’s lack of infrastructure in Long Island City,” she said.

The need for more classrooms could also have consequences for de Blasio’s push to make pre-K available to all the city’s 3-year-olds, an effort the city is rolling out slowly in part because of existing space constraints.

But even if thousands of students arrive with new Amazon employees, they will still represent only a drop in the bucket of the city’s 1.1 million public school students. De Blasio cautioned at Tuesday’s press conference that while some employees will live in the neighborhood, not all will move to Long Island City and some may commute from other areas. Still, the neighborhoods around Queens are some of the most crowded school districts in the city.

Concerns about the city’s record student homelessness

Seattle has struggled to address a surge in homelessness as home prices have soared more rapidly in the city than anywhere else in the country — an increase that many have attributed to its booming tech sector.

As the number of high-earners there has shot up, so has student homelessness, which has increased threefold between 2011 and 2017. But when the city tried to pass a new tax dedicated to boosting services for the homeless, Amazon led a campaign against the measure, which eventually died.

Amazon is promising to pay an average salary of $150,000 in New York City. In the school district that will host the tech giant’s new hub, about 72 percent of students come from low-income families.

In New York City, the number of homeless students is already at an all-time high. More than 114,000 students here lack permanent housing, which poses challenges for schools that may struggle to meet the needs of children who often lag behind their peers on academic measures.

Schools Chancellor Richard Carranza recently announced the education department would hire 100 new employees to help schools with high concentrations of homeless students.  

De Blasio said the arrival of large companies such as Amazon could exacerbate homelessness in other cities that “don’t have substantial affordable housing, are not building a lot of new affordable housing,” specifically calling out San Francisco.

But, he said, the impact of tax revenue from Amazon’s move will be “central” for supporting existing affordable housing in New York City.

Other changes in student demographics

School leaders in Seattle say the number of students who are learning English as a new language has jumped with Amazon’s growth, opening the need for teachers and curriculum to serve those students.

New York City has rapidly expanded its language programs under de Blasio, which are often seen as a tool to help spur more diverse schools. But the education department has also historically struggled to serve English language learners well.

Amazon’s move could have other effects on school diversity at a time when advocates have put increasing pressure on the the city to step up integration efforts. New York City schools are among the most segregated in the country, an issue that Carranza has pledged to tackle.

But with more higher-income families potentially lured to Queens by Amazon jobs, Cirrito worries about gentrification in a borough and neighborhood known for its diversity, and the effect that could have on classrooms.

“How can we say we welcome new Americans here if they can’t afford to live in Long Island City and they can’t afford to live in neighborhoods where their kids have good schools?” she asked. “At the time we have a chancellor in place calling for the desegregation of schools, this seems to be a move that will completely undermine his efforts.”

Even if low-income families live side-by-side with Amazon’s workers, it’s not at all clear their children will learn together. Long Island City is home to the New York’s largest housing project, and whether high-earners would opt into schools where many students are poor is an open question.

A philanthropic boost?

New York’s agreement with Amazon doesn’t offer many details about how the company will interact with the nation’s largest school system, but it does include a promise to create internships and “work-based learning opportunities” — including activities such as career days and mock interviews.

What that will look like, and whether a bigger stream of philanthropic support could follow, is unclear. Amazon has offered some support for public education in Seattle, including supplies for needy students. And its founder, Jeff Bezos, recently announced a $2 billion investment to launch a network of preschools in low-income communities.

Kathryn Wylde, the president and CEO of the nonprofit Partnership for New York City, which serves as the business community’s lobbying group, said she hopes Amazon’s presence helps fuel career and technical programs in city schools.

“The frustration has been a lack of employer engagement in opportunity for [career and technical education] and workplace opportunities,” Wylde said. “Obviously this is a bonanza in providing those opportunities.”

She added that Amazon could support schools similar to Brooklyn’s P-Tech, a high school that partners with IBM to offer students opportunities in the tech sector. (Wylde said there were no concrete plans in place yet for Amazon to participate in such a partnership.)

Others were less optimistic.

Alexander, the co-president of Community Education Council 30, said she hopes the city would partner “as much as possible” to harness any investments Amazon is willing to make in public schools.

Still, she added, “It sticks a little in my craw —  the richest person in the world getting billions of dollars in money from New York State when New York State owes schools so much money.”

“It’s hard to see what internship or guest speakers or whatever could make that balance.”

Chalkbeat live

Education for all? Let’s talk about that, Chicago.

Since Chalkbeat Chicago launched in June, we’ve convened small gatherings of parents, educators, school council members, and community leaders to talk about city schools.

On December 12, we’re hosting our biggest public forum to date — with pie! — on the topic of Chicago’s next mayor and the future of schools in the city.

  • Which items should top the next mayor’s schools agenda?
  • How do we build on successes like the district’s record-high graduation rates?
  • And how do we grapple with persistent challenges such as declining enrollment and equity gaps in performance and resources?

We are inviting educators, students, advocates, policy makers, and more to join us for this critical conversation. Taking part in the centerpiece panel will be Chicagoans with experience in and around schools including:

  • Jesus “Chuy” Garcia, founder of the Little Village Community Development Corporation and newly elected congressman from the 4th District
  • Elizabeth Swanson, vice president of strategy and programs at the Joyce Foundation and the former deputy chief of staff for education for outgoing Mayor Rahm Emanuel
  • Daniel Anello, CEO of Kids First, which produced a new report that examines school access and capacity on the city’s South and West sides
  • Jitu Brown, national director of the Journey for Justice Alliance

Guests will be encouraged to record a message to the city’s next mayor in our storybooth and to network with other people who care about public education and Chicago youth. The evening also will feature student performances and a coffee-and-pie reception with treats from Justice of the Pies and Dark Matter Coffee.

The event is free and open to all ages, but space is limited and registration is required. RSVP here.