Teaching teachers

‘Personalized learning’ comes to teacher training, bringing big ambitions and big questions

PHOTO: Woodrow Wilson Academy
Alex Trunnell collaborates with two other teaching "design fellows" at the Woodrow Wilson Academy of Teaching and Learning.

Imagine you’re a new teacher. You overhear two students disparaging Black Lives Matter protests, and know that other students heard it, too. You’re worried the comments will damage your classroom culture.

“What are you going to do in the exact moment? What do you do in the next month to make sure your classroom is a safe environment?”

Asking those questions is Rupal Jain of the Woodrow Wilson Academy of Teaching and Learning, a soon-to-launch graduate school of education with a new approach to teaching teachers. The Academy’s goal is not just to challenge them with scenarios like that one, but to ensure they master them, with prospective teachers moving at their own pace and graduating when they demonstrate more than 40 specific skills.

The future of education will “move away from focusing on what you’re being taught to what you’ve actually learned,” said Arthur Levine, the former president of Teachers College and the head of the Woodrow Wilson Foundation, the organization behind the Academy. “We thought, let’s create an institution that does it and can model it.”

The Academy, which will focus on preparing math and science teachers, is taking shape in partnership with MIT and with the support of major education funders. It recently netted a $3 million donation from the Chan Zuckerberg Initiative, the organization tasked with doling out the Facebook founder’s billions.

It amounts to a combination of two major efforts in American education: long-running attempts to improve teacher training to soften the on-the-job learning curve, and the newer effort to “personalize” education using technology and other means.

It’s unclear if it will work: “Competency-based” teacher education has a thin track record, and though research has been done on the teaching fellowships the Woodrow Wilson Foundation has run for the last decade, the foundation has not released it. But the Academy has the funding, prestige, and handle on the zeitgeist to suggest that its approach will influence teacher education in the years ahead.

What is the Academy?

Walk into the Academy headquarters today — an office in a nondescript building on MIT’s campus in Cambridge — and you’ll see evidence of furious brainstorming: Post-its, scribbled notes on whiteboards, a big concept map that staff members call their compass.

That work is a result of a partnership announced in 2015 between the Woodrow Wilson National Fellowship Foundation and MIT, which has attached an elite name to the endeavor and whose researchers are helping construct the curriculum.

The next two years were spent fundraising and sketching out how the program might work. Woodrow Wilson has raised $22 million from the Chan Zuckerberg Initiative, the Bezos Foundation, the Bill and Melinda Gates Foundation, and Carnegie Corporation of New York, among several other funders, and plans to raise $10 million more. (Chalkbeat is also supported by Gates and Carnegie.)

This year, 10 “design fellows,” mostly recent college graduates, are helping develop the program by serving as enthusiastic guinea pigs and idea generators.

The idea of building something new appealed to Alex Trunnell, who recently graduated from Vassar with a degree in physics and astronomy. Recently, she spent time trying to design ways to prepare teachers to ensure a classroom runs smoothly.

“How do you avoid any kind of hardship that isn’t grappling with the content?” she asked. “We realized that there is no right way to do those things. We can’t teach you the one right way to set up your classroom because it doesn’t exist.”

Instead, the Academy is creating a sort of teaching “gym” for aspiring teachers to practice, with activities and 3D software for designing a classroom space, for example.

The design fellows also visit schools once a week and work directly with students during after-school programs. And they’re using a simulation program known as “Mursion” for practicing classroom scenarios.

The inaugural class of of around 25 teacher candidates will start this fall. The Academy plans to ramp up to admit 50, then 75, and then 100 students by the 2021-22 school year.

But its goal is much larger in scope than producing new teachers. It’s to serve as a proving ground for a novel way of teaching teachers.

PHOTO: Woodrow Wilson Academy
Alex Trunnell, a teaching “design fellow” at the Woodrow Wilson Academy of Teaching and Learning.

The model’s challenges

To work, the Academy will need to successfully assess the skills it expects prospective teachers to master. That’s a tall order, particularly before teachers actually have their own classrooms.

Staff at the Academy say they plan to measure those skills repeatedly and in a number of ways, including written exams, virtual simulations of classrooms, and real-life student teaching situations. Still, certain context-specific skills, like being able to develop strong relationships with students, will always be challenging to gauge.

“We don’t have assessments yet that really assess the quality of those kinds of practices,” said Pam Grossman, the dean of the University of Pennsylvania school of education and a member of the Academy’s advisory board.

Another hurdle may be the funding model. Rather than charging based on the number of courses taken, the Academy plans to charge a set fee of $25,000 per student (with discounts based on student need) no matter how long it takes for someone to complete the program. The Academy will have to sell prospective students on that uncertainty — and keep students on track for its own financial sustainability.

“Every time I talk to my parents about this program it really freaks them out. It’s a hard thing to get your mind around, this idea that I don’t know when I’ll finish up,” said Trunnell. (She and the other design fellows will be able to enroll for free once the program launches later this year.) “For me, it’s actually really nice, because it’s this idea that I’m going to be done when I’m prepared and ready to be a good teacher.”

No research on Woodrow Wilson’s other teaching program

The Woodrow Wilson Foundation has been involved in teacher preparation for years, but its track record is unclear.

Its teaching fellowship, which launched in 2007, has partnered with universities in a number of states to train math and science teachers. Like the Academy, the fellowship aspired to “transform teacher education while preparing future leaders in the teaching profession,” according to its website.

We don’t know how well that effort worked, though. Despite contracting with the American Institutes for Research to study its fellowship, Woodrow Wilson has not released any external research about its fellowship programs.

A person with direct knowledge of a draft of a study of the fellowship in Michigan said it found that, on average, the Woodrow Wilson “fellows’ performance is about equal with the performance of comparable non-fellow teachers.”

The person spoke on the condition of anonymity because the findings are subject to a confidentiality agreement. (A spokesperson for Woodrow Wilson confirmed the existence of a nondisclosure agreement with AIR.)

Levine said he’s waiting for longer-term results from multiple states, and promised to release the research at some point in the future. Levine says that the Academy will also open itself up to careful study.

“What we’re waiting for is to bring this to a completion,” he said of the fellowship research. “I want real results before I start boasting or criticizing ourselves for them.” (The Foundation has, in two reports, though, claimed some success based on that research without releasing the full studies.)

The lack of publicly available information about the foundation’s long-running programs raises questions about the organization’s commitment to transparency.

Grossman, the University of Pennsylvania dean, said that, more broadly, it’s crucial to have careful studies on what is and isn’t working as teacher training programs try new things.

“We really need to be generating the research that adds to the knowledge base about what’s effective in teacher education,” she said. “And that means making the results of these studies public.”

If you build it, will others adopt it?

Let’s imagine that everything goes right with the Academy: it designs and executes its program well, it recruits full classes of new students each year, and it releases rigorous research showing that its graduates are successful in the classroom.

In that case, it will still be preparing just a hundred or so new teachers each year — even as public schools look to hire roughly 250,000 teachers annually and employ more than three million total teachers. To realize its goal, the Academy needs to be able to diffuse its approach widely.

Levine says that’s what they’re planning to do. “Everything we create is going to be open source,” he said. “The goal here is for this not be thought of as a competitor with traditional teacher ed providers — our goal is for this to be thought of as a resource center.”

That means some of Woodrow Wilson’s success will depend on whether the rest of the teacher prep world is interested in the Academy’s work and whether larger schools of education can put its simulations, games, and curriculum materials into use.

Ken Zeichner, a professor at the University of Washington who has been critical of some of the new teacher prep programs like Relay, said a lack of resources and expertise to implement a new approach had been the downfall of competency-based teacher education in the 1970s.

“These innovations are created, and you have all these universities that have not had the capacity to be able to implement the innovations that are being created,” he said.

There’s also the question of whether Levine — who has criticized existing teacher education programs for some time — is the right ambassador.

Levine says he’s not worried. “I may have been a critic, but I’m a critic who basically loves them,” he said. “There’s no example of us walking into an ed school and people saying, ‘Oh my God, you’re that monster!’”

Grossman says the Academy is more likely to be successful if it does not position itself as having all the answers. “Not, ‘We’re going to develop this and you can all learn from us,’ but ‘We’re in this together. We’re all trying to do some things to improve the quality of teacher education,’” she said.

fight another day

In union defeat, lawmakers end session without revamping teacher evaluation law

After a hard-fought battle by the state teachers union, New York lawmakers went home for the summer without overhauling a controversial teacher evaluation law that ties state test scores to educator ratings.

The bill pushed by the unions would have left decisions about whether to use state test scores in teacher evaluations up to local union negotiations. While the bill cleared the Assembly, it was bottled up by the Senate’s leadership, which demanded charter school concessions in return that Assembly Democrats wouldn’t agree to.

The effort to decouple test scores from teacher evaluations was one of several that fizzled out at the end of a lackluster session characterized by lawmaker gridlock.

“Sen. Flanagan, his caucus and five Democrats chose to betray the state’s teachers,”  said New York State United Teachers President Andy Pallotta in a statement. “Make no mistake, New York teachers, parents and public school students will remember which senators voted against their public schools when we head to the polls this September and again in November.”

There is some possibility that lawmakers could return to finish a few unresolved issues this summer, but Pallotta told Chalkbeat he is not holding out hope for that outcome.

The lack of action is a defeat for the state teachers union, which fought hard for the bill since the beginning of the session. Union officials have staged musical rallies, bought balloons, rented a truck with a message urging lawmakers to pass the bill, and capped off the last day of session handing out ice cream for the cause.

However, the legislative loss gives the union something to rally around during this fall’s elections. Also, other education advocacy organizations are content to engage in a longer process to revamp evaluations.

“Inaction isn’t always the worst outcome,” said Julie Marlette, Director of Governmental Relations for the New York State School Boards Association.“Now we can continue to work with both legislative and regulatory figures to hopefully craft an update to evaluations that is thoughtful and comprehensive and includes all the stakeholders.”  

The news also means that New York’s teacher evaluation saga which has been raging for eight years will spill over into at least next year. Policymakers have been battling about state teacher evaluations since 2010, when New York adopted a system that started using state test scores to rate teachers in order to win federal “Race to the Top” money.

Teacher evaluations were altered again in 2015 when Gov. Andrew Cuomo called for a more stringent evaluation system, saying evaluations as they existed were “baloney.” The new system was met with resistance from the teachers unions and parents across the state. Nearly one in five families boycotted state tests in response to evaluation changes and a handful of other education policies.

The state’s Board of Regents acted quickly, passing a moratorium on the use of grades three to eight math and English tests in teacher evaluations. But the original 2015 law remains on the books. It was a central plank in that law which could require as much as half of an educator’s evaluation to be based on test scores that the unions targeted during this session.

With the moratorium set to expire in 2019, the fight over teacher evaluations will likely become more pressing next year. It may also allow the state education department to play a greater role in shaping the final product. State education department officials had begun to lay out a longer roadmap for redesigning teacher evaluations that involved surveys and workgroups, but the legislative battle threatened to short-circuit their process.

Now officials at the state education department say they will restart their work and pointed out that they could extend the moratorium to provide extra time if needed.

“We will resume the work we started earlier this year to engage teachers, principals and others as we seek input in moving toward developing a new educator evaluation system,” said state education department spokeswoman Emily DeSantis.

For some education advocates, slowing down the process sounds like a good idea.

“Our reaction on the NYSUT Assembly teacher evaluation bill is that you could do worse but that you could also do better and that we should take time to try,” said Bob Lowry, deputy director of the New York State Council of School Superintendents.

What seems to be a setback for the union now may be a galvanizing force during elections this fall. Republican lawmakers will likely struggle to keep control of the state Senate, and NYSUT is promising to use this inaction against them. That could be particularly consequential in Long Island, which is a hotbed of the testing opt-out movement.

It’s unclear whether the failure to act will also prove problematic for Cuomo, who is also seeking re-election. Cuomo, who pushed for the 2015 law the unions despise, is facing competition from the left in gubernatorial challenger Cynthia Nixon.

But at least so far, it seems like the union is reserving the blame for Senate Republicans and not for the governor.

Cuomo is “making it clear that he has heard the outcry,” said Pallotta. “I blame Senator Flanagan, I blame his conference and I blame 5 [Senate] Democrats.”

a high-stakes evaluation

The Gates Foundation bet big on teacher evaluation. The report it commissioned explains how those efforts fell short.

PHOTO: Brandon Dill/The Commercial Appeal
Sixth-grade teacher James Johnson leads his students in a gameshow-style lesson on energy at Chickasaw Middle School in 2014 in Shelby County. The district was one of three that received a grant from the Gates Foundation to overhaul teacher evaluation.

Barack Obama’s 2012 State of the Union address reflected the heady moment in education. “We know a good teacher can increase the lifetime income of a classroom by over $250,000,” he said. “A great teacher can offer an escape from poverty to the child who dreams beyond his circumstance.”

Bad teachers were the problem; good teachers were the solution. It was a simplified binary, but the idea and the research it drew on had spurred policy changes across the country, including a spate of laws establishing new evaluation systems designed to reward top teachers and help weed out low performers.

Behind that effort was the Bill and Melinda Gates Foundation, which backed research and advocacy that ultimately shaped these changes.

It also funded the efforts themselves, specifically in several large school districts and charter networks open to changing how teachers were hired, trained, evaluated, and paid. Now, new research commissioned by the Gates Foundation finds scant evidence that those changes accomplished what they were meant to: improve teacher quality or boost student learning.  

The 500-plus page report by the Rand Corporation, released Thursday, details the political and technical challenges of putting complex new systems in place and the steep cost — $575 million — of doing so.

The post-mortem will likely serve as validation to the foundation’s critics, who have long complained about Gates’ heavy influence on education policy and what they call its top-down approach.

The report also comes as the foundation has shifted its priorities away from teacher evaluation and toward other issues, including improving curriculum.

“We have taken these lessons to heart, and they are reflected in the work that we’re doing moving forward,” the Gates Foundation’s Allan Golston said in a statement.

The initiative did not lead to clear gains in student learning.

At the three districts and four California-based charter school networks that took part of the Gates initiative — Pittsburgh; Shelby County (Memphis), Tennessee; Hillsborough County, Florida; and the Alliance-College Ready, Aspire, Green Dot, and Partnerships to Uplift Communities networks — results were spotty. The trends over time didn’t look much better than similar schools in the same state.

Several years into the initiative, there was evidence that it was helping high school reading in Pittsburgh and at the charter networks, but hurting elementary and middle school math in Memphis and among the charters. In most cases there were no clear effects, good or bad. There was also no consistent pattern of results over time.

A complicating factor here is that the comparison schools may also have been changing their teacher evaluations, as the study spanned from 2010 to 2015, when many states passed laws putting in place tougher evaluations and weakening tenure.

There were also lots of other changes going on in the districts and states — like the adoption of Common Core standards, changes in state tests, the expansion of school choice — making it hard to isolate cause and effect. Studies in Chicago, Cincinnati, and Washington D.C. have found that evaluation changes had more positive effects.

Matt Kraft, a professor at Brown who has extensively studied teacher evaluation efforts, said the disappointing results in the latest research couldn’t simply be chalked up to a messy rollout.

These “districts were very well poised to have high-quality implementation,” he said. “That speaks to the actual package of reforms being limited in its potential.”

Principals were generally positive about the changes, but teachers had more complicated views.

From Pittsburgh to Tampa, Florida, the vast majority of principals agreed at least somewhat that “in the long run, students will benefit from the teacher-evaluation system.”

Source: RAND Corporation

Teachers in district schools were far less confident.

When the initiative started, a majority of teachers in all three districts tended to agree with the sentiment. But several years later, support had dipped substantially. This may have reflected dissatisfaction with the previous system — the researchers note that “many veteran [Pittsburgh] teachers we interviewed reported that their principals had never observed them” — and growing disillusionment with the new one.

Majorities of teachers in all locations reported that they had received useful feedback from their classroom observations and changed their habits as a result.

At the same time, teachers in the three districts were highly skeptical that the evaluation system was fair — or that it made sense to attach high-stakes consequences to the results.

The initiative didn’t help ensure that poor students of color had more access to effective teachers.

Part of the impetus for evaluation reform was the idea, backed by some research, that black and Hispanic students from low-income families were more likely to have lower-quality teachers.  

But the initiative didn’t seem to make a difference. In Hillsborough County, inequity expanded. (Surprisingly, before the changes began, the study found that low-income kids of color actually had similar or slightly more effective teachers than other students in Pittsburgh, Hillsborough County, and Shelby County.)

Districts put in place modest bonuses to get top teachers to switch schools, but the evaluation system itself may have been a deterrent.

“Central-office staff in [Hillsborough County] reported that teachers were reluctant to transfer to high-need schools despite the cash incentive and extra support because they believed that obtaining a good VAM score would be difficult at a high-need school,” the report says.

Evaluation was costly — both in terms of time and money.

The total direct cost of all aspects of the program, across several years in the three districts and four charter networks, was $575 million.

That amounts to between 1.5 and 6.5 percent of district or network budgets, or a few hundred dollars per student per year. Over a third of that money came from the Gates Foundation.

The study also quantifies the strain of the new evaluations on school leaders’ and teachers’ time as costing upwards of $200 per student, nearly doubling the the price tag in some districts.

Teachers tended to get high marks on the evaluation system.

Before the new evaluation systems were put in place, the vast majority of teachers got high ratings. That hasn’t changed much, according to this study, which is consistent with national research.

In Pittsburgh, in the initial two years, when evaluations had low stakes, a substantial number of teachers got low marks. That drew objections from the union.

“According to central-office staff, the district adjusted the proposed performance ranges (i.e., lowered the ranges so fewer teachers would be at risk of receiving a low rating) at least once during the negotiations to accommodate union concerns,” the report says.

Morgaen Donaldson, a professor at the University of Connecticut, said the initial buy-in followed by pushback isn’t surprising, pointing to her own research in New Haven.

To some, aspects of the initiative “might be worth endorsing at an abstract level,” she said. “But then when the rubber hit the road … people started to resist.”

More effective teachers weren’t more likely to stay teaching, but less effective teachers were more likely to leave.

The basic theory of action of evaluation changes is to get more effective teachers into the classroom and then stay there, while getting less effective ones out or helping them improve.

The Gates research found that the new initiatives didn’t get top teachers to stick around any longer. But there was some evidence that the changes made lower-rated teachers more likely to leave. Less than 1 percent of teachers were formally dismissed from the places where data was available.

After the grants ran out, districts scrapped some of the changes but kept a few others.

One key test of success for any foundation initiative is whether it is politically and financially sustainable after the external funds run out. Here, the results are mixed.

Both Pittsburgh and Hillsborough have ended high-profile aspects of their program: the merit pay system and bringing in peer evaluators, respectively.

But other aspects of the initiative have been maintained, according to the study, including the use of classroom observation rubrics, evaluations that use multiple metrics, and certain career-ladder opportunities.

Donaldson said she was surprised that the peer evaluators didn’t go over well in Hillsborough. Teachers unions have long promoted peer-based evaluation, but district officials said that a few evaluators who were rude or hostile soured many teachers on the concept.

“It just underscores that any reform relies on people — no matter how well it’s structured, no matter how well it’s designed,” she said.

Correction: A previous version of this story stated that about half of the money for the initiative came from the Gates Foundation; in fact, the foundation’s share was 37 percent or about a third of the total.