With all the focus on accountability at the DOE - progress reports, data collection, learning environment surveys, quality reviews, and more - I find myself frustrated at the gaps in timely, useful data made available to the public by the DOE. Accountability has to go in both directions - teachers, parents, and all citizens should have access to data quickly, presented in ways that allow independent analysis. With mayoral control up for renewal in the spring, with new initiatives such as the "Million" motivation plan, with increasing emphasis on standardized tests and regular interim assessments, with data collected from the Learning Environment Surveys being used to judge schools, we have more reasons than ever to want to see for ourselves whether DOE policies and practices have been fair and effective. Yet data is not easy to find.
Yesterday, I took an initial look at the Manhattan Institute's study, "Building on the Basics." Today, I want to look at Florida's state science exam, the focus of the study. A common criticism of standardized tests is that they all, to some degree, test reading ability. What does the Science FCAT look like? What skills would you need to perform well on it? I've only seen the NYS Science exams, so I decided to download a Florida sample test and take a look. The first thing that surprised me about this test was the reading level, which seemed high. Many of New York City's fifth graders would (for better or for worse) stumble over sentences like, "Florida has many limestone caves containing formations called stalactites." I tracked down a site of readability analyzers and entered text from test items. Question 1: Melissa’s school rings a bell to alert students that it is time to start class. When the bell rings, it vibrates. The use of vibrations to send messages is an example of which type of energy? This one ranged from 4.72 to 10.07 in estimated US grade level required to understand it, which certainly calls into question the reliability of the readability analyzers, but also the ability of average 5th graders to understand this question.