What would Santa Fe Public Schools like to do with its lowest-achieving students?
Make them disappear, according to a report on student achievement data that has become the latest flash point between SFPS Superintendent Bobbie Gutierrez and the current SFPS Board of Education.
As SFR previously reported, the Superintendent's Report to the Board of Education that Gutierrez produced Feb. 17 contains student achievement data that doesn't match up with official numbers published by the state Public Education Department. Although SFPS initially gave a completely different explanation for the discrepancy, SFPS Director of Assessment and Accountability Lynn Vanderlinden now tells SFR that the district's report differs from PED numbers because SFPS used only positive data, removing data on students whose proficiency declined.
The Superintendent's Report purports to show the academic improvement of SFPS students from the 2008-2009 school year to the 2009-2010 school year. But it's apparent just by looking at one category of students—for instance, economically disadvantaged elementary school kids—that something in the report doesn't quite compute. According to Standards Based Assessment test score data available from PED, reading proficiency among that category of students went from 36.06 percent to 36.3 percent over the year. But the Superintendent's Report quantifies that small change as an 8.89 percentage-point leap.
"That's probably because we were only looking at areas of growth, rather than areas of growth and loss," Vanderlinden says of the report's data.
Vanderlinden explains that the data was created by removing all of the instances where proficiency decreased, and tabulating only the increases in proficiency. The students with negative changes in proficiency were simply zeroed out.
Vanderlinden tells SFR that she devised this method of presenting the achievement data after observing that the district's numbers for Annual Yearly Progress (an analysis of the SBA data used to evaluate schools and districts) "weren't great" and "looked depressing."
"When AYP comes in, I receive the data and of course prepare the data to hand to the superintendent," Vanderlinden says. "I looked and they weren't great numbers. So I thought, 'Well, right now, I'm only seeing AYP, so I want to see if there were any areas of improvement.' And so we sat down and we combed through looking for improvement, and that was the report we created."
Creating a report designed to downplay "depressing" academic achievement is hardly an effective way to address the district's shortcomings, Fred Nathan, founder and executive director of Think New Mexico, a Santa Fe think tank with a focus on improving education, says.
"The first step in solving the serious problems with SFPS student achievement is honestly assessing the data," Nathan writes SFR in an email. "I agree with the central administrative office that the numbers are 'depressing,' but that is not a reason to actively deceive the governing board—and by extension, the public—about them."
PED spokesman Larry Behrens says districts should present complete data.
"The expectation will always be that districts use accurate and complete numbers from all students to educate parents," Behrens says.
Not only does the idea of simply eliminating unfavorable data fail the sniff test; it's also at odds with the explanation the district originally gave for the data discrepancy. In May, SFPS Deputy Superintendent Mel Morgan told SFR that SFPS' data didn't match up with PED's because the agencies group grade spans differently, with SFPS sometimes including sixth grade as elementary school instead of as junior high. But SFPS data is based on the PED data, which clearly defines grade categories consistently. Morgan could not be reached for comment all last week and early this week.
Vanderlinden emphasizes that she created the controversial report on her own initiative, without a directive from Gutierrez or anyone else at SFPS. She says it was her idea to use only the positive numbers. Gutierrez says Vanderlinden and Morgan created the report and she wasn't involved.
However, meeting minutes available on SFPS' website show that former Board Member Angelica Ruiz asked Gutierrez at a Feb. 2 board meeting to "put together something with her mid-year responses to some of the expectations they laid out to her for this year." The Superintendent's Report that apparently fit that bill was created Feb. 17—the same day the board evaluated Gutierrez' performance and renewed her employment contract during an executive session.
The board laid out those expectations for Gutierrez in September 2010; the second of seven goals was to increase student achievement. Had the Superintendent's Report presented the full picture of student proficiency, it wouldn't have given Gutierrez much to brag about. Instead of significant increases in proficiency, the report would have shown modest (1 or 2 percentage point) increases or, in some cases, decreases in proficiency.
At a June 21 meeting, Board Vice President Glenn Wikle called the report's representation of student achievement data "very misleading" and made a motion for a more complete analysis of the data to be completed and presented to the board.
The motion passed, but Vanderlinden is the one charged with this task.
"It is completely unacceptable for the same employee who admitted manipulating the data to conduct an audit of her own calculations," Nathan writes. "Parents, students and teachers deserve to have an independent, outside expert examine the numbers and provide an accurate report on student achievement in Santa Fe."
Board Member Steven Carrillo also tells SFR he would prefer to have an outside agency go over the data, but what he really wants is an explanation of why the data was manipulated that way and why skewed data was used as a "basis for evaluation" for Gutierrez' job performance.
"All they did was look for places where there were increases, and that to me was a fairly nonsensical was to try to get a picture of the overall achievement of the district," Carrillo says. "That is ridiculous to me—to try to grade the district for how well we've done, but all you're looking at is a few pockets where we know the numbers went up. We're actively and knowingly omitting all the numbers where it decreases."
Gutierrez says she believes she needs clearer communication with the board in future regarding what type of report it wants to see.
"When it comes to data, and this is one thing that in retrospect and in moving forward needs to happen, there's a lot in how you ask the question," Gutierrez says. "I think we need to be sure that we're asking the right questions and that this is indeed the information [the board wants]."
Neither Carrillo, Wikle nor current BOE Secretary Linda Trujillo were on the board in February when it voted, based partly on the Superintendent's Report, to renew Gutierrez' contract. Former Board Member Richard Polese tells SFR that, to the best of his recollection, the board was not told that there was any unusual method of calculation used in creating the achievement data contained in the Superintendent's Report.
"As far as my recollection goes, we were not told it was done in a specific way or in some kind of special presentation that was incomplete," Polese says. "I don't recall being told that."
To some SFPS parents, though, data that leaves out the lowest-achieving students is definitely "incomplete."
"[Gutierrez] calculated in a zero for every actual decrease in student proficiency of a subgroup," SFPS parent Cate Moses tells SFR. "That means she was literally erasing the students we should care the most about, the ones that are falling behind."