As the city pushes to expand out-of-school time programming for middle schoolers, a summer camp with a mixed record will grow significantly next month.

NYC Summer Quest, a free, five-week program for low-income elementary and middle schoolers, has impressed parents and increased students’ preparation for school in the fall, according to surveys conducted by the Department of Education. Chancellor Carmen Fariña has repeatedly advocated for its growth as an antidote to summer learning loss, the phenomenon in which students—especially from low-income backgrounds—regress academically over summer vacation.

But pre- and post-program testing for middle schoolers revealed poor academic results in year one and inconclusive results in year two. As the program enters the final summer of its three-year pilot and the city prepares to add 1,000 spots exclusively for middle school students, Summer Quest is facing an important test.

“We all intuitively think there’s a lot of value, but to really show the academic effects, I think you need two to three cycles,” said Margaret Crotty, executive director of Partnership with Children, which has run a Summer Quest program for the past two years. “Our hope is that the third year is really when you see the gains.”

This year, Summer Quest will work in Brooklyn and the south Bronx to provide reading and math instruction and camp-style activities to students who might otherwise have little to do over the summer. Most of these students performed well enough on state exams to move on to the next grade, though others will be allowed to participate as an alternative to summer school.

The number of campers has more than doubled since 2012, growing from 1,100 students in year one to 2,800 this summer. The program’s price tag this year is $6.2 million, with approximately $2.3 million coming from private sources and the rest funded by the Department of Education and the Department of Youth and Community Development.

Each Summer Quest site is run through a partnership between a school and a community based organization. These partnerships are given a great deal of autonomy; while the department encourages project-based learning and provides a framework for Summer Quest’s curriculum, daily activities can range from yoga to drumming lessons to cooking classes.

Despite these differences, department evaluations have shown certain trends. In the first year of the program, 80 percent of parents felt their children were better prepared for school after completing the program.

But department pre- and post-assessments showed that Summer Quest did not stem summer learning loss for middle school students—though that age group is Fariña’s focus and the target of the current expansion.

It is unclear if the academic results improved in year two, because the city did not administer the same assessments in 2013, which Crotty said was part of an effort to minimize the number of tests given to middle schoolers.

Instead, the department compared the middle school students’ springtime state exams with their performance on the summer school test, which Summer Quest campers took at the end of the program, according to a department official.But the state exams and Summer Quest academics are Common Core-aligned, and the summer school exam is not, making for a less accurate comparison.

The department has yet to announce how it will evaluate students’ academic progress this summer.

If successful, Summer Quest has the potential to narrow the achievement gap, which research shows is significantly affected by cumulative summer learning loss. While some Summer Quest sites evaluate students throughout the following school year, the department does not conduct its own long-term assessments, although doing so might provide insight into the program’s impact on the achievement gap.

But Summer Quest isn’t all academics. The program also seeks to improve students’ self-confidence, social-emotional skills, and connection to their schools.

In 2013, 96 percent of parents said the program increased their child’s self-confidence in the classroom. The same percentage wanted to send their children back to Summer Quest the following year.

Activities at Summer Quest can take a variety of forms. Building Educated Leaders for Life, a CBO that works at three different schools, implements a camp theme called “My Passport to the World.”

“I worked with the principals to develop a program that would expand the vision for these young people,” said Timothy Coleman, BELL director of field operations. In one activity, BELL partnered with nonprofit Reach the World to allow campers to Skype with graduate students in various parts of the globe. In real time, graduate students showed campers how to order coffee at a French café.

At the Wingspan/Urban Assembly School for Applied Math and Science site, Summer Quest students go on overnight camping trips in the Black Rock Forest. Students hike into the woods, pitch their tents, build a campfire, and participate in team-building activities.

The trip “gives most students their first experience of being out in the wilderness,” Wingspan executive director Paul Ashley said.

Department officials say they have continued to refine Summer Quest over the past two years, partially in reaction to the weak academic results from year one. The department has gathered feedback from students, parents, and teachers, and sites also receive qualitative assessments from the National Summer Learning Association.

Asia Franks, a ninth grade integrated algebra teacher from the Urban Assembly School for Applied Math and Science, is also the instructional coach for the school’s Summer Quest site. She said that feedback has led her site to incorporate more reading and writing instruction into the arts over the last few years.

National research on summer learning time has also informed changes. For example, after the RAND Corporation found that successful summer programs include a minimum of 15 hours a week of teacher-led instruction, the city made those 15 hours an explicit requirement at all sites.

Department officials say they will focus on this summer before announcing plans for Summer Quest’s future beyond its three-year pilot.

“This third year is an opportunity for the sites to just knock it out of the park,” Partnership with Children Program Director Alex LaMond said. “We want to be able to establish, empirically, that it does impact summer learning loss.”

This story is part of a multi-city series on expanded learning time, with funding from the Ford Foundation, which supports “more and better learning time” in high-need communities. Participating along with Chalkbeat’s four bureaus in the series are the Notebook (Philadelphia), Catalyst Chicago, and EdSource (California).

Want the latest in New York City education news? Subscribe to our Rise & Shine newsletter.