paulracko / Pixabay
Some education “reformers” sing the praises of using Value Added Measurements in evaluating the performance of teachers.
However, researchers have found many reasons to be cautious about using them. I’ve posted about them at The Best Resources For Learning About The “Value-Added” Approach Towards Teacher Evaluation, including:
NEW STUDY QUESTIONS USE OF VAM IN TEACHER EVALUATION
WHAT A SURPRISE (NOT!) – NEW STUDY FINDS THAT VALUE-ADDED APPROACH TO TEACHER EVALUATION ISN’T ACCURATE
New Study Finds VAM Is Biased Against Teachers Of “At Risk” Students
Then, of course, there are other studies questioning the whole role of test scores in teacher evaluation, and you can find those posts at The Most Important Studies Showing That VAM For Test Scores Is Not The Be-All, End-All Of Schooling.
Now, yet another study pokes hole in the infallibility of VAM.
The Sensitivity of Value-Added Estimates to Test Scoring Decisions finds that:
While the various test scores are highly correlated, on average, using different scoring approaches leads to variation in VA percentile ranks of over 20 points, and more than 50% of teachers or schools are classified in multiple quartiles of the VA distribution. Dispersion in VA ranks is reduced with more complete item response data. Our findings suggest that consideration of both measurement error and model uncertainty are important for the appropriate interpretation of VAMs.
I asked ChatGPT to put that in non-academic language:
Here is a simpler version in plain language:
Even though different test scores usually line up pretty closely with each other, the way you calculate “value-added” scores (how much a teacher or school is said to improve student learning) can change the results a lot.
On average, using different calculation methods can shift a teacher’s or school’s ranking by more than 20 percentile points. In fact, over half of teachers or schools end up in different performance groups (like top 25%, middle, or bottom 25%) depending on which method is used.
The rankings become a little more stable when there is more complete and detailed test data about students.
Overall, the study is saying:
Value-added scores are not perfectly precise. There is both measurement error (tests aren’t perfect) and model uncertainty (different formulas give different answers). So these scores should be interpreted cautiously, not treated as exact or absolute measures of teacher or school quality.
Source link
#Study #Finds #Reasons #Cautious #Added #Measurements #Evaluate #Teachers
