When Idaho lawmakers voted in 2015 to increase teacher pay by more than $250 million, many had no idea the accountability measures they attached to their funding formula were rooted in flawed data.
Working as part of a two-person reporting team at Idaho Education News, we used public school records to show that school administrators gave every teacher from 32 of the state’s 115 school districts identical evaluation scores.
The reporting process consumed nearly five months; we submitted our first public records request to the State Department of Education Jan. 26 and published our story June 11.
To explain how we did it, I need to back up and give credit to my partner on the project. Our editor, Jennifer Swindell, created a new position and hired longtime Idaho school superintendent Randy Schrader as our data and policy analyst.
Schrader had experience submitting accountability reports to the state as part of his former job as superintendent.
Teacher evaluation scores (not the entire evaluation portfolio, but the overall scores) are public in Idaho and other states.
During the 2015 legislative session, state lawmakers passed a law to increase teacher pay tied to accountability.
The biggest accountability provision was a teacher’s evaluation. In order to move from one rung on the salary schedule to the next (and thus earn higher pay) teachers needed to meet evaluation performance benchmarks.
Idaho evaluations are scored on a scale of one (unsatisfactory) to four (distinguished).
Schrader filed a written public records request with the State Department of Education’s PIO, Kelly Everitt, under Idaho’s Public Records Act.
We requested evaluation scores broken down by each of Idaho’s 115 districts and charter schools. We asked for the actual number of teachers, by district, who received scores in each of the four evaluation categories.
Everitt had been just hired by newly-elected State Superintendent of Public Instruction Sherri Ybarra, and Schrader had to guide him throughout the five-month reporting process.
We requested electronic copies of the records, and there was no charge for fulfilling the request since there were no physical copies to print. We file multiple public records requests with the State Department of Education each month (always seeking electronic copies only) and have never been charged to fulfill a request.
“I received the initial data and reviewed it and immediately knew it was wrong,” Schrader said. “Many districts submitted the same data for all teachers and, overall, the state had no teachers as unsatisfactory and very few rated basic.”
Schrader also shared the data with a former State Department of Education employee, who confirmed his suspicions.
The data indicated that none of the state’s 17,365 teachers received evaluation scores of “unsatisfactory,” and only 151 teachers received “basic” scores. Everyone else earned scores of “proficient” and “distinguished.”
We used basic Excel functions to sort the data, add up statewide totals and determine percentages for each category.
That’s when Schrader noticed another red flag: the total number of evaluations in the state’s report did not correspond to the total number of teachers.
The PIO told Schrader certain scores were redacted to protect teacher privacy.
The PIO also said he lost our first records request.
Schrader filed a second public records request, but the state returned different data even though our requests were identical. We retained our original records and again used basic Excel functions to calculate totals and percentages, allowing us to compare the two sets of records we received from the state. The PIO attempted to argue that he was allowed to redact numbers in instances where we could add a school district’s totals from all four categories together to equal 100 percent.
In order to convince Everitt to release accurate and complete data, Schrader confronted him with a relevant section of the state’s administrative rules.
It worked, and Everitt released a report with the totals of all the rows and columns matching the total number of teachers. The new report also included “unsatisfactory” scores that were absent from the initial evaluation report.
But we were still suspicious. In 32 school districts, every teacher earned the same evaluation score of “proficient.” That means, according to official state reports, none of those teachers in any of those districts received evaluation scores of “distinguished,” “basic,” or “unsatisfactory.” The report also indicated that just 33 of the state’s 17,000-plus teachers earned scores of “unsatisfactory.”
Armed with the data, I confronted school superintendents about the numbers and questioned the accuracy of the reports. Many of them dismissed my questions, responding that everybody already knew that Idaho employs great teachers. Nothing to see here, move along.
Then I showed the data to Ryan Kerby, an Idaho lawmaker and retiring superintendent of a small school district. Kerby awarded identical evaluation scores to all 59 of his teachers. As a member of the Idaho House, he also voted in favor of the new salary law that relied on teacher evaluations as an accountability measure to tie taxpayer dollars to teacher salaries.
Here is what he said, on the record.
“Our school district, quite unanimously, did not figure the state needs to know all that individual teacher data. We feel the state should be concerned with whether kids are learning, not if Mrs. Smith got proficient or unsatisfactory or basic… It’s really not something that the Legislature should be concerned with every Mrs. Smith in fourth grade in New Plymouth.”
Now we had solid data and a great story. We cleaned up the data in Excel for ease of presentation, sorting rows by school district. Then we imported our data into Google Sheets, added a drop-down search function dialogue box to allow readers to isolate any individual school district. We also presented a table with results from all 115 school district and embedded it into WordPress to display it alongside our story.
The story quickly jumped to the top of our most read articles of the month. The governor’s press secretary, Jon Hanian, responded in a follow-up interview, saying: “The data regarding trends in teacher evaluations highlights the need for ongoing training for administrators in how to conduct the evaluations as they should be done.”
The State Department of Education was already preparing a committee to audit teacher evaluations, but state officials added to the scope of their work after the publication of our story, asking the committee to review evaluations practices overall.
In a follow-up story, the president of the Idaho Association of School Administrators, Alan Dunn, confirmed that his district also submitted erroneous evaluation data to the state.
“I feel confident most, if not all, of the districts that gave everybody the same evaluation, those scores did not reflect and did not correlate with the actual teacher evaluations they gave,” Dunn said.
Dunn purposely turned in results to the state indicating all his teachers received identical scores, even though the actual evaluations on file in his office told a different story.
Finally, a member of the Legislature’s budget committee, which controls the purse strings for teacher pay, said she plans, based on our findings, to probe and review the salary law and accountability measures during the 2016 legislative session.
Clark Corbin is a reporter for Idaho Education News. You can contact him at firstname.lastname@example.org and follow him on Twitter at @clarkcorbin.