From Voodoo to Statistics. . .

At a dinner I was recently a part of, Robert Eaker—Rick DuFour’s co-author and colleague—said this:

"The range of teacher strategies for identifying student learning difficulties across a hallway in a single school building runs from voodoo to statistics."

His central point resonated with me because I know that I’ve personally struggled with assessment of student learning for years. 

Even though I’m considered to be a highly accomplished teacher, I’m not sure I can reliably judge the strengths and weaknesses of the students in my classroom.  I honestly suspect that "measurement error" (in the lingo of those driven by standardized testing) is probably quite high in my room–both in my comparisons between students and in my comparisons of multiple tasks submitted by the same student. 

Do you think that statement is true for your classroom?  How about your grade level or school?  If it is true, how does that reality impact your efforts to help every child learn?  How does it harm the standing of our profession in the eyes of the general public?  Does it cheapen who we are when our responses aren’t–in some degree—standardized and automatic?

Have you seen a change over time in the kinds of strategies used by teachers to identify student learning difficulties?  If so, what’s caused that change—and is it a change that we should embrace as a profession?  Do your colleagues embrace it?

Final question:  Have you ever felt fully prepared to identify student learning difficulties?  What was the key to that preparation?  Has that confidence and ability ever been effectively spread across your entire building?

Looking forward to your responses!

3 thoughts on “From Voodoo to Statistics. . .

  1. Mike

    Dear Bill:
    I’ve pondered this one for a few days, and realized that I’ve been pondering it to death. I, and every other competent teacher, knows how to evaluate student learning for simple, easily understood reasons, which include: education, training (past and current), teaching experience, maturity, life experience, and common sense. If this was not so, how could the education system function at all? How could it be that the American education system has produced the most productive, well educated, technically competent citizenry and nation in world history, much of that progress occurring since 1945? In addition, if we assume that tests, high stakes-mandatory or otherwise have power superior to that of teachers to reveal student learning, who, pray tell, was able to write those tests and what magical powers beyond those of teachers do they possess? Weren’t these superior, test-writing beings taught by the very teachers who lack the ability to determine whether the future test writers were learning? Let’s not even get into the issue of politicians and others claiming that the public education system is turning Americans into drooling morons without explaining how they and they alone escaped its dumbifying effects.
    And no, there is no voodo involved. The evaluation process begins with daily lesson planning. Of course there is variability within the process from teacher to teacher. Not every teacher is a great teacher; not every teacher works as hard, but that’s the nature of humanity. As long as everyone involved meets minimum acceptable standards, that’s OK and is truly the best we can expect. Is it possible to have a school filled with nothing but the best possible teachers? Possible, but not likely. I don’t suggest we shouldn’t search for ways to improve, merely that we shouldn’t beat ourselves up over much simply because we’re not perfect. We’ll reach perfection tomorrow.
    But back to daily plans. We choose lessons and materials that are appropriate for the grade levels of our students and that will help them to improve. We do this knowing that not every student in a given class is at grade level in every facet of their learning. We know that we must take students where they are when they walk into our classrooms on the first day of school and take them as far down the road as time, our abilities and their effort will allow in a year. We can do this because we are experienced, we have training, we are mature and we have common sense. Do we grade each and every paper according to a strict rubric demanding that each and every student meet a specific standard? No. We sometimes grade according to individual progress and the need to encourage or reprove. And we know to do this for the aforementioned reasons.
    Can I tell when a given student is working hard on a given assignment compared to when they barely try? Sure. Can I tell when a student works hard, even if there are still a great many errors in a writing? Yes, and for all the aforementioned reasons. Do I sometimes make measurement errors? Give an undeservedly high grade or an unjustifiably low grade? Yes, but fewer than teachers with less experience, and I encourage my students to discuss such things with me when they feel I’ve made an error. It they’re right, or at least have a good, logical argument, I’ll make changes, and that’s a good part of the process of learning too. Again, the point is not that I’m perfect but that I’m good and so are most teachers. And perfect is the enemy of good, and good is good in any human endeavor. We can always do better, but good is pretty good, and isn’t that well and good? Good.
    We hire plumbers because they have knowledge and abilities we lack, not because we believe that they will be perfect, but because we believe that they will do well. The same is true of teachers.
    Our assessment of student learning is, in many ways, consistent across grades, disciplines and teachers, but it is not machine-like and cannot be quantified in the same way that a data set from a standadardized test can be quantified. All good teachers have many of the same habits and methods and they are not diminished because they are not identical and applied in identical ways. What, after all, is a better and more accurate source of the abilities and accumulated knowledge of a given student? The teacher who has worked with that student for an entire year, five days a week over 150+ assignments of all kinds, or a single test score? And as to diminishing our standing with the public, you need merely ask them what I just asked you. Who would have more and more accurate knowledge about you, the teacher who worked with you daily for a year, or anyone with one test score in their hand? Should anyone answer the latter, you should immediately rap them sharply upside the head and charge them to get ahold of themselves. The public understands the value of good teachers and knows all too well that they know what progress is and isn’t. They just don’t think of it in those terms all the time, but they remember Mr. Smith who wouldn’t let them get away with less than their best, or Mrs. Jones who always called them on laziness and encouraged them.
    Since much of what I’ve pointed out happens more or less automatically, day in and day out, it’s not something most teachers discuss. But it is passed on through mentoring and informal teacher chatting, sharing and instruction.
    And yes, I’m prepared to identify student learning difficulties (I’m not talking about the work of a diagnostician here), and the key to that is all of the education, experience, maturity and trial and error that led me to this point in my life. I know I can do better; I know I make mistakes, but I lead an examined life and I try to improve and avoid hubris. I also try not to flog myself and others with too many wet noodles.
    You notice that nothing I’ve said was in eduspeak. You notice that it wasn’t research-based. You notice that even the public at large could understand it. If we want the public at large to understand and appreciate us, that may be part of what we have to do–speak to them in understandable, jargon-free ways. And stop whacking ourselves with wet noodles.
    If you really, upon reflection, don’t believe that you, after all your years of experience, can’t reasonably judge student progress, put down your wet noodle and take a manufacturing job where you either put the widget into the slot correctly or didn’t. If that’s not the case, keep living an examined life and strive for perfection tomorrow, but suck it up and keep striving no matter what. Do you really think that the people who replace you with less education, knowledge and experience will do better?

  2. Bob

    Yes, approaches, rhetoric, tactics, and interpretations of and about student learning in classrooms has changed over the past four or five decades. I think you know this. Also, most teachers know that fundamental facts about how people learn and what constitutes learning have not changed beyond refinements to principles in experimental learning research achives. And, teachers still fuss among ourselves about the same kinds of issues concerning learning and our possible parts in urging it to occur. I look forward to neuropsychologists contributing further refinements and possibly new principles of learning teachers may use to increase learning rates.
    As to measurement error, most people in and out of education know it occurs routinely, and that’s why professionals try to account for it objectively in order to reduce it.
    Did I address your main question?

Comments are closed.