prep problems

A new study shows why it’s so hard to improve teacher preparation

Dramatically reshaping how teachers are trained — by emulating great teacher preparation programs and shutting down ineffective ones — has been a key priority of many states and even, under the Obama administration, the federal government.

Fierce debates have ensued over how to hold training programs accountable for making sure novice teachers are ready for the classroom on day one.

Now a new study casts doubt on those efforts for a simple reason: It’s hard to identify good or bad teacher preparation programs, at least as measured by student achievement.

That’s the provocative conclusion of research by Paul von Hippel of University of Texas at Austin and Laura Bellows of Duke University.

“It appears that differences between [programs] are rarely detectable, and that if they could be detected they would usually be too small to support effective policy decisions,” write von Hippel and Bellows.

The study, which has not been formally peer-reviewed, follows other peer-reviewed research comparing teacher training programs in Florida, Louisiana, Missouri, New York City, Texas and Washington.

Those studies try to isolate the impact of teacher preparation programs — including schools of education and alternative certification initiatives — on student test scores. It’s a difficult task, since there are two degrees of separation between a teacher training program and students taking state tests, and researchers use complex value-added models to control for a number of factors.

The studies come to differing conclusions. Some suggest that programs vary substantially in effectiveness, but most find few clear differences among a state’s programs.

In the latest research, von Hippel and Bellows reanalyze those six studies using a consistent method. They find that in all states the differences between teacher preparation programs are small and it’s difficult to pick out top-notch programs with confidence.

“This is troubling because singling out [programs] is a prerequisite to the policy goal of expanding strong [programs] and shuttering weak ones,” they write.

Indeed, a number of states — 16, according to the researchers — have made efforts to evaluate training programs based in part on value-added measures. The Obama administration issued regulations encouraging states to evaluate their teacher prep programs by measuring student learning, though they were scrapped by Congress and the Trump administration earlier this year.

Dan Goldhaber, who has studied Washington’s teacher training programs and is a professor at the University of Washington, said this latest study is consistent with his own findings.

“I think there’s relatively little variation in the value-added effectiveness of teachers who hold credentials from different programs,” he said.

Still, he notes, the size of the impact is in the eye of the beholder. At most, the difference between attending a good versus an average training program is comparable to the difference in effectiveness between the average first- and third-year teacher — definitely not big, but not necessarily zero.

An inherent limitation of this research is that it focuses exclusively on the fraction of teachers who end up in tested grades and subjects, largely fourth- through eighth-grade math and English.

It may be more helpful to judge teacher preparation programs by multiple measures. For instance, recent research has found that there is substantial variation in how different programs affect teachers’ scores on classroom observations, which can be used to evaluate all teachers, not just those in tested areas.

Still, isolating the impact of training remains a challenge, since teachers are not randomly assigned to schools, and some programs aim to place teachers in high-poverty schools where attaining high ratings may be more difficult.

Bellows, the Duke researcher, warned against chasing after programs that might be appear effective simply because of a statistical fluke.

“You don’t want to remodel our [teacher preparation programs] based on one that looks really good when … it’s just by chance,” she said.

However, Bellows and von Hippel’s research suggests that there is some reason for optimism. In five of the six states, there was at least one large program that appears significantly better at preparing teachers in at least one subject.

“If we are very careful, we can occasionally identify a [program] that is truly exceptional,” they write.

negotiations

Aurora school board reverses course, accepts finding that district should have negotiated bonuses with union

Students in a math class at Aurora Central High School in April 2017. (Photo by Yesenia Robles, Chalkbeat)

Following weeks of criticism, the Aurora school board on Tuesday reversed course and accepted an arbitrator’s finding that a pilot bonus system violated the district’s agreement with the teachers union.

The Aurora school district rolled out an experiment last year to offer bonuses to some teachers and other staff in hard-to-fill positions, such as psychologists, nurses and speech language pathologists.

The teachers union argued that the plan should have been negotiated first. An arbitrator agreed and issued a report recommending that the pilot program stop immediately and that the district negotiate any future offerings. The union and school board are set to start negotiations next month about how to change teacher pay, using new money voters approved in November.

When school board members first considered the arbitrator’s report last month, they declined to accept the findings, which were not binding. That raised concerns for union members that the district might implement bonuses again without first negotiating them.

Tuesday’s new resolution, approved on a 5-1 vote, accepted the full arbitrator’s report and its recommendations. Board member Monica Colbert voted against the motion, and board member Kevin Cox was absent.

Back in January 2018, school board members approved a budget amendment that included $1.8 million to create the pilot for incentivizing hard-to-fill positions. On Tuesday, board member Cathy Wildman said she thought through the budget vote, the school board may have allowed the district to create that incentive program, even though the board now accepts the finding that they should have worked with union before trying this experiment.

“It was a board decision at that time to spend that amount on hard-to-fill positions,” Wildman said.

Board president Marques Ivey said he was not initially convinced by the arbitrator’s position, but said that he later read more and felt he could change his vote based on having more information.

Last month, the Aurora school board discussed the report with its attorney in a closed-door executive session. When the board met in public afterward, it chose not to uphold the entire report, saying that the board could not “come to an agreement.” Instead board members voted on a resolution that asked the school district to negotiate any future “long-term” incentive programs.

Union president Bruce Wilcox called the resolution “poorly worded” and slammed the board for not having the discussion in public, calling it a “backroom deal.” Several other teachers also spoke to the board earlier this month, reminding the newest board members’ of their campaign promises to increase transparency.

Board members responded by saying that they did not hold an official vote; rather the board was only deciding how to proceed in public. Colorado law prohibits schools boards from taking positions, or votes, in private.

The board on Tuesday also pushed the district to provide more detailed information about the results of the pilot and survey results that tried to quantify how it affected teachers deciding to work in Aurora.



story slam

The state of teacher pay in Indiana: Hear true stories told by local educators

It’s time to hear directly from educators about the state of teacher pay in Indiana.

Join us for another Teacher Story Slam, co-hosted by the Eiteljorg Museum of American Indians and Western Art, Chalkbeat Indiana, and Teachers Lounge Indy. Teacher salaries are the hot topic in education these days, in Indiana and across the country. Hear from Indianapolis-area teachers who will tell true stories about how they live on a teacher’s salary.

Over the past two years, Chalkbeat has brought readers personal stories from the teachers, students, and leaders of Indianapolis through our occasional series, What’s Your Education Story? Some of our favorites were told live during teacher story slams hosted by Teachers Lounge Indy.

Those stories include one teacher’s brutally honest reflection on the first year of teaching and another teacher’s uphill battle to win the trust of her most skeptical student.

Event details

The event will be held from 6-8 p.m. on Friday, March 15, at Clowes Court at the Eiteljorg, 500 W Washington St. in Indianapolis. It is free and open to the public — please RSVP.

More in What's Your Education Story?