Taking a hit

By getting testing wrong again, Tennessee could undo what it may be getting right

PHOTO: TN.gov
Gov. Bill Haslam attends a Knoxville school assembly in 2016 to celebrate Tennessee outpacing almost all other states in gains on a national science exam administered by the National Assessment of Educational Progress.

After languishing for decades, Tennessee is now considered a pioneer in education improvement circles, while its standing on national tests has risen from the bottom to the middle of the pack over the last decade.

Quick to embrace higher academic standards, the state also explored new strategies to transform struggling schools and adopted a controversial teacher evaluation system grounded in student performance. With its 2010 federal Race to the Top award, it poured millions of dollars into teacher training and coaching.

So it was dumbfounding this spring when a third straight year of problems emerged with TNReady, the standardized test that’s the centerpiece of Tennessee’s policy agenda aimed at becoming a national leader in student achievement.

After a failed rollout of computerized testing in 2016 and scoring issues in 2017, the stakes had never been higher to administer the test successfully.

But on the very first day, thousands of students struggled to advance past the login page — and things went downhill from there. Subsequent testing days were marred by a cyber attack, a fiber optic cable cut by a dump truck in rural Tennessee, and a systems design error that made 1,400 students take the wrong exam.

By the time the state limped across the finish line last week, technical glitches had interrupted more than half of the original testing days, and state lawmakers had passed emergency legislation weakening how the results can be used.

“We’ve failed significantly with TNReady — not once, not twice, but three times,” said Rep. Craig Fitzhugh, a Democratic candidate for governor and member of a House education committee. “It hurts systems that are already beset with credibility problems.”

Now, many stakeholders worry that public outcry over this year’s sloppy testing will unravel years of carefully crafted accountability work in public education.

Rep. Eddie Smith of Knoxville stands at the front podium of the Tennessee House of Representatives on April 25, the last day of the 2018 legislative session, as the chamber’s education leaders press for a bill to hold teachers harmless for this year’s TNReady scores.

An early sign came when lawmakers stepped in before testing was even finished to shield students, teachers, schools, and districts from the state’s score-driven accountability systems.

“It happened so quickly and passed with such a large majority that it was jarring to those of us who thought we had some serious momentum with the systems we’ve been working on for years,” said Gini Pupo-Walker, leader of the Tennessee Educational Equity Coalition and a member of the state’s testing task force.

“It’s created this environment of instability going into a gubernatorial election year, and you begin to wonder what else could be wiped away.”

Politics of education

Over the last 16 years, Tennessee has managed to follow the same general roadmap for improving its schools — first under the administration of Democrat Phil Bredesen and now under Republican Bill Haslam.

But this year’s near meltdown of testing has put Haslam’s administration on the defense in the governor’s final year in office.

Education Commissioner Candice McQueen and her team are trying to see if the test results are valid and how they can be used. They’re also in daily talks with the U.S. Department of Education over whether the state’s emergency TNReady laws conflict with a federal law that requires student achievement-based accountability.

Haslam will do damage control this week after returning from an overseas economic development trip. He’s expected to beat the drum about the role of a state test as a measuring stick to ensure that children are learning and taxpayers get their money’s worth for the billions spent on schools.

McQueen offered an early glimpse at the messaging late last week. She said this year’s poor delivery of a computerized exam under testing company Questar does not mean that the exam itself is bad.

"We need to continue on the path that we’re on because we are much closer to success than not."Candice McQueen, commissioner of education

“We have an exceptional test,” McQueen told Chalkbeat on Friday, adding that the fix needs to come with how testing is administered. She added that it would be a mistake to change course on the state’s education agenda.

“The three things that we have focused on — high standards, rigorous assessment, and greater accountability — have been the backbone of much of our success in Tennessee,” she said. “We need to continue on the path that we’re on because we are much closer to success than not.”

Tension with testing

Indeed, national test results have been encouraging. Tennessee’s ACT average finally hit a modest milestone last year, and scores on several national tests are up since 2011. Just months ago, the state was basking in the glow of a massive Stanford University analysis showing Tennessee’s academic gains have outpaced the rest of the South — and much of the nation.

But testing that exceeds federal requirements has taken its toll on school communities, partly because districts have introduced extra exams to make sure students and teachers are on target leading up to TNReady.

PHOTO: TN.gov
Education Commissioner Candice McQueen has been under fire for her oversight of the state’s standardized test, which has had a string of high-profile problems since its 2016 rollout.

“I think we’ve gone dramatically overboard with testing,” said Dan Lawson, a superintendent in Tullahoma. “Everybody felt a very heavy hand on them when it came to this year’s assessment.”

That tension bubbled up last month during legislative hearings amid the online testing interruptions.

“What we have created, I’m afraid, is a culture of testing instead of a culture of teaching,” Rep. Sheila Butt told McQueen.

The Republican from Columbia went on to read a letter from one teacher: “I’m not sure this year if we’re actually wanting academic accountability,” the teacher wrote, “or if we’re merely testing our students’ resilience in the face of obstacles and our teachers’ patience with the new system.”

Sen. Dolores Gresham, who chairs her chamber’s education committee, offered another viewpoint.

“I remember a retired teacher telling me one time that she just wanted to be left alone, close the door, and she would go into her classroom and teach. And I wanted to tell her that that’s how we got to be 46th in overall student achievement,” said the Somerville Republican, “because we did not know what was going on in that classroom.”

“We have got to know,” Gresham concluded. “We have got to be able to evaluate and know what to do going forward.”

Joshua Glazer, a professor and researcher at George Washington University, said it’s understandable that frustrations with TNReady could amplify concerns about testing in general. But he cautioned against any knee-jerk reaction that minimizes assessments.

“We haven’t gotten the testing and accountability thing right yet, for sure,” said Glazer, who has followed Tennessee education policy. “But that doesn’t mean we want to go back to the 1980s when everybody could do whatever they wanted and we saw massive inequality in opportunity as a result.”

For more on how Tennessee got here, check out why state lawmakers share blame, too, for TNReady testing headaches.

right to know

A bastion of student data privacy, Colorado yields a bit to demands for more openness

PHOTO: Getty Images

Colorado education officials are reconsidering the data privacy rules that for three years in a row have hidden large amounts of student achievement data from public view.

With Thursday’s release of state test results, the public has greater ability to see how well certain groups of students perform on state tests compared with their peers than they’ve had since 2015, when the state adopted a much more stringent approach.

The main change this year is that the public can see results separated by race and ethnicity, disability status, English language learner status, and economic status at the school level, rather than only by grade levels within schools. Combining results across grades creates larger sample sizes, so that in some cases they no longer had to be redacted. The practice of obscuring results for small groups of students had led to large amounts of missing information and vexed advocates for school improvement.

And state education officials are considering additional changes for 2019 that should make more information available – though just how much remains to be seen.

“In this latest release, we were very pleased to be able to see how kids are making progress,” said Van Schoales, executive director of A Plus Colorado, an education reform advocacy group that focuses on research. “We can now see if they met or exceeded standards for groups of 16 or more. We thank the Colorado Department of Education for figuring out some methods for showing that.”

A Plus Colorado was part of a coalition that wrote the Colorado Department of Education last November calling for more transparency. That group includes the Colorado Children’s Campaign, the business-oriented education reform group Colorado Succeeds, civil rights groups like Together Colorado and Padres y Jóvenes Unidos, Democrats for Education Reform, the Colorado League of Charter Schools, and the Colorado Freedom of Information Coalition.

“The current lack of transparency in student data and results is a major challenge to our collective efforts to build a better education system,” they wrote. “Results that are not transparent and comprehensive not only undermine our ability to improve students’ education, but call into question the honesty and integrity of the entire accountability system.”

Advocates and state education officials met through the spring and summer to see if any common ground could be found. The state also hired an outside firm, HumRRO, to assess its data practices.

That firm, according to Deputy Commissioner Alyssa Pearson, found that Colorado was right to change its practices in 2015 but that there was room for greater transparency while still protecting student privacy.

This debate doesn’t involve test results of individual students but rather information about how groups of students at particular schools perform. Advocates for school accountability say the ability to know, for example, how well students with disabilities or those learning English perform in comparison with their peers is key to knowing how well schools are serving all their students.

Colorado was previously one of the most open states in providing detailed information about student performance. But starting in 2015, the Colorado Department of Education withheld a lot more information, particularly when results were separated out by categories of students. Instead of hiding results just for groups of 16 students or smaller, the state hides results from additional categories of students to prevent a careful observer from working backward to determine what was in the redacted field. This is known as complementary suppression.

And instead of showing how many students scored in each tier on state tests, Colorado shows what percentage of students met or exceeded expectations. That makes it hard to know if students who didn’t meet expectations were just below the threshold or way at the bottom.

Colorado plans to keep obscuring results from small groups of students and to continue the practice of complementary suppression. There are still missing results in the 2018 data publicly released so far. However, by combining results from different grade levels within schools, a more complete picture emerges of how each subgroup of students is being served.

No single state or federal law dictates the specific practices that Colorado has adopted. In fact, federal law requires that states make disaggregated data about student performance available to the public, so long as it doesn’t end up revealing personally identifiable information. These practices are what state officials decided was necessary to prevent the release of information traceable to individual students. That provides some wiggle room to reassess those practices.

Pearson said state officials are focused on who needs to know what information and for what purposes and then trying to accommodate that within privacy requirements. They’re also revisiting the question of how likely it is a “reasonable person” could trace certain pieces of data back to individual students.

In addition to making detailed school-level data available, Colorado has revamped its own online search tool, Data Lab, to make it easier for the public to search for how different groups of students performed at particular schools. Right now the information is available across multiple spreadsheets available for download. The data from 2018 tests should be incorporated into Data Lab sometime in September.

Another change on the table: Reducing the threshold for suppressing data. State officials will decide by the end of the year about whether to lower the bar for concealing group data to 10 students, from the current 16. That would mean less data would be suppressed – but how much less isn’t clear yet.

Pearson said the state hopes to make a final decision by the end of the year so that changes can be incorporated into the 2019 testing cycle. It hasn’t been decided yet whether administrators would make that decision on their own or whether the State Board of Education would vote on the policy change.

Along with pressing the state to release more information, advocates for more transparency have also launched a Right to Know website and campaign to encourage districts and schools to share more information with families. Districts are required by law to share information from state assessments with parents, but practices vary widely across the state.

At a recent meeting of the State Board of Education, members of the Colorado Youth Congress, part of the Right to Know coalition, urged the state to do more to put this information in the hands of the people most affected.

“As a student, I want to know if my school is performing strongly or not so I can understand the setting I’m in and if I want to change to a different setting that would benefit me more,” said Jasmine Kabiri, a junior at Silver Creek High School in Longmont. “In a different situation, if we’re shown the performance data of my school, I could help motivate my school toward advancing themselves.”

School choice

Secret CPS report spotlights big vacancies, lopsided options for students

The school district says the report will help inform how it invests in and engages with communities. Communities groups worry the document will be used to justify more school closings, turnarounds and charters.

An unreleased report by a school choice group backed by the business community paints in stark detail what many Chicagoans have known for years: that top academic schools are clustered in wealthier neighborhoods, and that fewer black and Latino students have access to those schools.

The report highlights startling figures: About 27 percent of black students are in the district’s lowest-rated schools, compared with 8 percent of Latino students and 3 percent of whites. It also says that while Chicago Public Schools has more than 150,000 unfilled seats, 40 percent, or 60,000 of them, are at top-ranked schools. That surplus will grow as enrollment, which has been plummeting for years, is projected to decline further by 5.1 percent over the next three years. What that means is the cash-strapped district is moving toward having nearly one extra seat for every two of its students.

The document effectively shows that, in many areas of the city, students are skipping out on nearby options, with less than half of district students attending their designated neighborhood schools.

In a city still reeling from the largest mass school closure in U.S. history, this report could lay groundwork for another round of  difficult decisions.

The “Annual Regional Analysis” report, compiled by the group Kids First Chicago on CPS’ behalf, has been circulating among select community groups but has not been made public. It comes on the heels of a report showing students’ high school preferences vary with family income level. Students from low-income neighborhoods submit more applications than students from wealthier ones and apply in greater numbers for the district’s charter high schools.

The group behind the latest report has had many iterations: Kids First is a new name, but its origins date back to 2004, when it started as the charter fundraising group Renaissance Schools Fund. That was during the Renaissance 2010 effort, which seeded 100 new schools across the city, including many charters. The group changed its name to New Schools Chicago in 2011 and again rebranded this year as Kids First, with a greater focus on parent engagement and policy advocacy.

The report has caused a stir among some community groups who’ve seen it. Because the school district has used enrollment figures to justify closing schools, some people are worried it could be used to propose more closings, turnarounds, and charter schools.

“To me this is the new reason [for school closings],” said Carolina Gaeta, co-director of community group Blocks Together, which supports neighborhood schools. “Before it was academics, then it was utilization, now it’s going to be access and equity. Numbers can be used any way.”

In a statement on the report, Chicago Teachers Union Spokeswoman Christine Geovanis blasted Mayor Rahm Emanuel’s administration for policies that she alleged “undermine enrollment at neighborhood schools,” such as the proliferation of charter schools, school budget cuts, and building new schools over the objection of community members.

Reached by phone Thursday, Kids First CEO Daniel Anello confirmed that his organization helped put the report together, but declined to comment on its contents, deferring to the district. CPS Spokeswoman Emily Bolton acknowledged the report’s existence in a statement emailed to Chalkbeat Chicago that said the school district “is having conversations with communities to get input and inform decisions” about where to place particular academic programs. The statement said CPS is still in the process of drafting a final version of the document, but gave no timetable. Mayor Rahm Emanuel’s office didn’t grant requests for interviews about the Annual Regional Analysis.

Below is a preview of the report provided to Chalkbeat Chicago.

Gaps in access to arts and IB programs

Data released this week from the district’s GoCPS universal high school application clearly shows what academic programs are most in demand: selective enrollment programs that require children to test in;  arts programs; and career and technical education offerings, or CTE.

The Kids First’s analysis puts those findings into context, however, by detailing how supply is geographically uneven, especially when it comes to arts. Maps in the report divide the city into regions defined by the city’s planning department and show how highly-desirable arts programs are not spread equally throughout the city, and are most concentrated along the northern lakefront and downtown.

PHOTO: Sam Park
This map shows the number of fine & performing arts program seats available per 100 elementary school students in each planning area.

Worse, four regions offer 10 or fewer arts seats per 100 students, including the Bronzeville/South Lakefront region that includes neighborhoods such as South Shore, Woodlawn, Kenwood and Hyde Park. They are also scarce in the West Side region, which includes Austin, North Lawndale, and Humboldt Park and in the Northwest neighborhoods of Belmont Cragin, Dunning, and Portage Park.

The report also shows an imbalance in the number of rigorous International Baccalaureate programs.

This map shows the number of IB program seats per 100 students available to elementary and high school students in each planning area.

The highest number of IB seats are in the wealthy, predominately white Lincoln Park area. In contrast, there are far fewer IB seats in predominantly black communities such as  Englewood and Auburn Gresham, Ashburn and in the predominantly Latino Back of the Yards.

When it comes to selective-enrollment elementary school programs such as gifted centers and classical schools, which require students to pass entrance exams, options tend to be concentrated, too, with fewer choices on the South and West sides of the city. This map shows where selective enrollment high school options are most prevalent:

PHOTO: Sam Park
This map shows the number of selective enrollment high school seats available per 100 students in the city’s planning regions.

STEM programs are more evenly distributed across Chicago than both IB and selective enrollment schools, yet whole swaths of the city lack them, especially on the South Side, including the Greater Stony Island. As the other maps show, that region lacks most of the high-demand academic programs the district has to offer.

PHOTO: Sam Park
This map shows the number of STEM program seats available per 100 elementary school students.

Racial disparities in school quality

The analysis also shows disparities in quality of schools, not just variety.

At CPS, 65 percent of students districtwide are enrolled at Level 1-plus or Level 1-rated schools. But only 45 percent of black students and 72 percent of Latino students are in those top-rated seats, compared with 91 percent of white students.

The disparities are even more severe given that the school district is mostly Latino and black, with fewer than one in 10 students identified as white. 

A page from a presentation of the Annual Regional Analysis showed to select community groups.

In the Greater Lincoln Park region, 100 percent of elementary schools have one of the top two ratings — the highest concentration of them in the city.  The highest concentration of top-rated high school seats, 91 percent, is in the Central Area, which includes Downtown and the South Loop.

The lowest concentration of top-rated elementary seats, 35 percent, is in the Near West Side region, and the lowest concentration of high school seats, 14 percent, is in the West Side region.

Long commutes from some neighborhoods

The number of students choosing schools outside their neighborhood boundaries has increased in recent years.

But the report shows that school choice varies by race: 44 percent of black students attend their neighborhood elementary school, compared with 67 percent of Latino students, 69 percent of white students, and 66 percent of Asian students. For high schoolers, only 14 percent of black students attend their neighborhood school, compared with 28 percent of Asians, 30 percent of Latinos, and 32 percent of whites.

More students enrolling outside their neighborhood attendance boundaries means more and more students have longer commutes, but how far they travel depends on their address. 

Again, this is an area where the Greater Stony Island area stands out.

A graphic from the Annual Regional Analysis executive report that shows how far elementary school students in each of the city’s 16 planning regions travel from their homes to school. The data shows that students on the South and West Sides tend to have longer commutes.

The average distance traveled for elementary school students is 1.5 miles — but K-8 students in Greater Stony Island travel an average of 2.6 miles. The average distance to class for high schoolers citywide is 2.6 miles, but students in the Greater Stony Island region travel an average of 5 miles, about twice the city average. 

A graphic from the Annual Regional Analysis executive report that shows how far high school students in each of the city’s 16 planning regions travel from their homes to school. The data shows that students on the South and West Sides tend to have longer commutes.

Looking forward

The introduction to the Annual Regional Analysis describes it as “a common fact base” to understand the school landscape. It clearly states the intent of the report is to assist with district planning, not to provide recommendations.

It still bothers Wendy Katten, founder of Raise Your Hand, who has seen the report and said it tells little about how kids are actually learning at schools.

“It sounds like some data a company would use to reduce inventory at a manufacturing plant,” she said.

Gaete with Blocks Together said the numbers in the report are also missing important context about how the proliferation of charter schools, a lack of transparent and equitable planning, and a lack of support for neighborhood schools in recent decades has exacerbated school quality disparities across race and neighborhoods in Chicago, one of the nation’s most diverse but segregated cities.

It’s unclear when the final study will be published, or how exactly the school district will use its contents to inform its decisions and conversations with communities.

But an event posting on the website for Forefront, a membership association for “nonprofits, grantmakers, public agencies, advisors, and our allies,” mentions a briefing for the report on Oct. 10.

Kids First Chicago CEO Dan Anello and CPS Director of Strategy Sadie Stockdale Jefferson will share the report there, according to the website.