The American Statistical Association (ASA), an organization of professionals whose jobs are to make sense of numbers, warned two years ago of the unfairness and inaccuracy inherent in using test scores to evaluate teachers, but some educational bureaucrats still refuse to listen.
I looked up the ASA report again after Education Commissioner Mike Morath approved a new teacher evaluation model under which school districts could base 20 percent or more of a teacher’s evaluation on “student growth measures,” including so-called value added measures (VAM).
A VAM model typically is based on a complicated formula that compares a group of students’ actual scores on standardized tests to scores predicted by an equation based on test scores of other, but similar student groups. It is an opaque process that is incomprehensible to most educated people, but the American Statistical Association has figured it out and raised warning flags that Morath has failed or refused to see.
TSTA believes that Morath also has exceeded his authority as education commissioner in proposing that element in his teacher evaluation plan, and we have sued him to try to keep it from going into effect.
In its assessment, released in April 2014, the American Statistical Association warned that using VAMs for teacher evaluations could be counterproductive because the practice could result in even more class time being spent on test preparation “at the exclusion of content that may lead to better long-term learning gains or motivation for students.”
ASA also noted that, based on most VAM studies, teachers account for only about 1 percent to 14 percent of their students’ variability in test scores.
“The majority of the variation in test scores is attributable to factors outside of the teacher’s control, such as student and family background, poverty, curriculum and unmeasured influences,” the association wrote in its report.
Teachers welcome fair, productive evaluations that encourage professional development. They deserve to be evaluated on more than a numbers game, and the numbers experts agree.