Rejecting "Evidence-Based Practice"

There are many words and phrases I'd love to toss out of our educational vocabulary. "Rigor" is, of course, a big one. "Flipped Classroom" is another - unless we're going to really try the old Fred Astaire-Royal Wedding thing. "Special Needs," "Special Education," and "SpEd" are others because it is way beyond time to embrace Universal Design for Learning, full inclusion, IEPs for everyone, and Toolbelt Theory for all. But here I am with one more... the dreaded "Evidence Based Practice."

"Evidence Based Practice" sounds so good, sounds so authoritative, sounds so "medical." It is, after all, medical. The term was "Evidence Based Medicine," and like the "Gold Standard" research promoted by the AERA and the US Department of Education, it sounds great to the uneducated, but remains not just impossible to transfer to the field of education, but undesirable.

I heard a medical researcher on NPR this week describing how a highly effective counter-treatment to the side effects of antibiotics couldn't really spread because it could not be evaluated the FDA "gold standard" way, and it reminded me of how evaluation systems stymie innovation everywhere.

But even if these research and practice paradigms worked fully in medicine... we still could not, and should not, import them. First, it is not possible to do a "double blind" trial in education, the practitioner surely knows what "treatment" is being used, and so do most students (we often might as well give out "Control Group" T-shirts) so the "Gold Standard" promoted by AERAis simply fraudulent. Then there's the undesirable - we are educators, it is malpractice for us to withhold interventions from students when we believe they will help.

And so if the methods of collecting our "evidence" are both fraudulent and irresponsible... where does that leave "Evidence Based Practice"?

It leaves it as the tool of those who want to preserve - to harden - the industrial-process nature of education, and as the tool of hucksters with products to sell, like researcher-salesmen Slavin and Marzano. As Peter Smagorinsky says, in terms of how the US Department of Education operates:

 "Consider the ways universities operate. Universities initially compete for students, but once they’re enrolled, that game is over. After enrollments are secured, universities compete for prestige, and the game is not zero-sum: The more prestigious universities, the better the nation is served. The U.S. university system is not centrally organized, so each campus competes in its own way. Public universities are bound by state regents systems, at times intrusively; but for the most part are entrusted to be the institution they want to be. Innovations are very public and available as models for others to consider. Indeed, the publishing imperative forces good work out into the open and contributes to the status of the institution as others seek to adopt it. This approach works well for universities, but public school teachers are not entrusted with the same opportunity.

"The competition and innovation that local control would promote would enable teachers to exercise judgment, an opportunity that is rarely available under the current testing regime. Just as importantly, it would help to shape the incentive system within which education takes place. Presently, young people looking for a field of endeavor read daily attacks on teachers and schools, and see a punitive, assembly-line approach to teaching that follows from standardized curriculum, instruction, and assessment imposed by Washington bureaucrats with no teaching experience. Replace that image with a workplace where teachers’ ideas matter and their innovations are appreciated as dynamic and worthy of admiration. Which environment would you decide to enter if you were a talented young person looking for a rewarding career? Most teachers I know say that they want to teach because they want to make a difference. It’s hard to make a difference when the system rewards those who make everything the same." - Peter Smagorinsky, Distinguished Research Professor of English Education at The University of Georgia and included in Valerie Strauss' Washington Post blog The Answer Sheet, March 28, 2012

I am not against "evidence," we all collect "evidence" to support our actions or to suggest change in our actions every day, but I am also for innovation and creation and experimentation every day, especially in schools. For example, I have never repeated the same "class" - not even when teaching the same thing three times in a row. I learn from each presentation and then adapt, that's my evidence based practice, you know, "well, that didn't work," but there is something else creating the change - the "audience," the interactions change every time - and I can not let "evidence" from a previous experience interfere with the need to innovate, improvise, based on the needs currently before me.

This is the problem with "Evidence Based Practice," with "Common Core," with any enforced "National Standards," all limit our possibilities by enforcing "what is" as Gary Stager noted on Twitter: "All capitulations to national standards reduce your ability to make decisions as a teacher." All capitulations to the "scientific evidence" requirement prevent innovation.

What we need instead in education is a nation, nations, of experimental researchers working constantly with Grounded Theory based Action Research. All teachers as innovative experimenters, all schools as laboratories of what might be possible. This is essential, because otherwise we are, to use the metaphor for this week, simply re-arranging the deck chairs on the Titanic because we saw them arranged differently on the Central America.

In other words, our current praxis, our commitment to "Evidence Based Research," is utilizing fraudulent research paradigms to investigate micro-changes in a system developed not through science or research at all, but as a Social Darwinistic effort to prevent social change which might threaten power structures. [See the Five Part series - Designed to Fail.]

Grounded Action

"If a grounded theory is one that, as Strauss (1990) contends, “…is inductively derived from the study of the phenomenon it represents," then grounded action could be described as “…one that is inductively derived from the study of the phenomenon it represents for the purpose of creating and applying practical solutions to social systems” (Simmons & Gregory, 2003)."

Putting together Grounded Theory [link is to a pdf], the observation-based data collection concept which accepts that the real world is more powerful than the laboratory for social structures,  with Action Research, the continually adaptive method of applying developing knowledge to the solution of problems in addition to the study of them, allows us to form a circle which moves us - and more importantly - our students forward. Forward into a world wholly unrelated to the Henry Barnard - Ellwood Cubberley structure of education which remains intact even when "flipped" or "flattened" or privatized.

"Operational theory is systematically generated from and grounded in explanatory theory. The operational theory provides a grounded theoretical foothold for action planning and implementation."

You learn to see your classroom anew. You observe all the possible options. That becomes your data floor. You make changes, observing the results, constantly observing, recording, and adapting to the inflow of information. You are educators. You are not going to trap a control group of children into a failed reading system for 5 months in order to prove that system works - you are going to keep searching for solutions for those kids - but you are going to do it like researchers... recording your observations, your changing efforts, and your successes and failures, and then sharing that with the world of education - wholly and freely.

And if we all work on invention, and we all share what we're doing, and we're all building on what we're doing, we can stop buying third rate adaptations and begin to grasp the possible future.

Which has to be our goal. To move rapidly into the future, where our students already reside. And we're not going to get there buying ten-year-old crap, that we collected "evidence" for five-years ago.

- Ira Socol

"Acting Student" and the issue of "Confirmation Bias"

I got into a couple of battles with Stuart Buck on Twitter recently. Stuart is the author of Acting White: The Ironic Legacy of Desegregation, but when we first clashed, I did not know that. He was arguing that not teaching students what I'd call "traditional skills" was wrong.

It was a heated conversation, but when I paused to think, I thought, "Who is this guy?" So I Googled him, and I found his book, and I bought it. I tend to think this is an important "academic" idea - the close reading of those you disagree with. Sometimes it changes your mind in important ways - I know that, for example, my thoughts on Horace Mann changed dramatically as I really read his words when I began writing my thesis. Sometimes, certainly, it doesn't. But either way, you need to try.

Lincoln School, New Rochelle, NY - the evidence might suggest that when integration
has not been undone, there are differing results.

I don't think he bought mine. Right, but I'm a curious sort of person.

Life Magazine, 1966
Having lived through a big part of this era, I might begin with different questions.

Buck's book, published by the Yale University Press, claims to be a deep reading of the school desegregation experience in the United States and its long term impacts on African-American attitudes toward success in school. He can be an entertaining writer, and at times he writes a good story. And it is not a "one-note" story either, as Richard Thompson Ford wrote in Slate, Buck, "tiptoes through the minefield with nuance and compassion. He credibly (and repeatedly) insists that he supports school desegregation but wants to be forthright about its unintended consequences, so we can find ways to contain them."

And yet, as Ford says (and I have attempted to say to Buck via angry Twitter argument), "But Buck's focus on schools neglects the bigger picture. The power of the epithet "acting white" is just one manifestation of a belligerent youth subculture among poor blacks that rejects mainstream institutions generally. "Acting white" is to education as "stop snitching" is to law enforcement: an attitude of aimless and self-destructive opposition, borne of deprivation, alienation, and despair. The root cause lies in the depth and pervasiveness of inner-city poverty. Buck argues that poverty can't be the cause of "acting white" because "blacks in the Jim Crow era … pursued education eagerly even in the presence of far more dire poverty. If poverty … caused the 'acting white' criticism, it surely would have shown up long before the 1960s." But the problem isn't just objective poverty. It's also social isolation, which worsened dramatically at precisely the time Buck says the acting-white problem emerged."

Buck does not, in my mind, see the big picture precisely because he fails, at the start, to ask the right questions. And perhaps - and I know I'm treading into dangerous assumption territory here - he doesn't ask the right questions because he knew what he wanted to write before he began his research.

What leads me to believe this begins with Buck's casually negative characterizations of people he clearly sees as opponents, those who, "argue that that many of the things learned in school - ways of thinking, writing, or problem-solving, for example - are "white" or "Eurocentric" (p. 19)," as political and sloppy (he quotes William Darity as describing his research as "warfare in the academy" (p. 20) in an attempt to discredit his work at the start).

Obviously, though, Buck's book is essentially a political, not an academic, argument. This is a free-market, America "as it is" declaration, insisting that African-American students and African-American communities must change, because we don't want to (or cannot) make any changes in America's socio-economic or educational structure. Ford again, "Buck endorses everything from vouchers to exclusively black-male charter schools to the novel idea of replacing individual grades in integrated schools with academic competitions between teams of students."

Thus, it is not really a surprise that Buck thinks that when I suggest that we rely on student strengths, and adapt curriculum and especially communication methods to student strengths, we are cheating kids. Or that he gets angry when I suggest that our schools are "white," "Eurocentric," places obsessed with creating "norms." I say these things in a way which suggests problems, and Buck, the white adoptive father of two Black children (book flap), has a very large personal and cultural stake in cultural conversion and moving all toward "a norm."

My experience - and politics - differ. Having watched school desegregation proceed clumsily and badly in New Rochelle, NY, I have also seen what happens when integration has continued in a single public high school city with consistent support for educational institutions and facilities. So, seeing differing results in differing circumstances I might begin from a less certain place.

And thus I might ask different questions up front, including the questions Buck has told me - via Twitter - that he had no interest in asking. Questions about the purpose of the American education system, questions about cognitive theories, questions about learning theories. Even questions about term definitions - how, as an example, might one define, "thinking, writing, [and] problem-solving"? What, I might ask, is, "Acting Student"? And if "Acting Student" is "Acting White," what does that suggest? Because I would ask, can you really discuss these issues if you do not begin by making sure that everyone understands what you mean when you say something?

I think you do. Unless your goal is to confirm what you thought in the first place. Perhaps if your solution is 'changing the students' - your research will lead you to believe that is necessary.

And Buck's book suggests to me that he "neglects the bigger picture" because the bigger picture clashed with the hypothesis - and perhaps a very personal hypothesis - he began with. And not just his book, in a later "battle" we argued over the definition of "private school" - and whether the US definition was the same as the European definition. Buck showed little understanding of the concept of state religion or the differing social structures - perhaps (again, that dangerous assumption) because admitting that Catholic schools in the Republic of Ireland are not really equivalent to for profit charter schools in the United State would challenge the political narrative (defunding American public education) embraced by both himself and his Ivy League sponsors.


Confirmation Bias is a huge problem in research, especially social science research. We too often begin with an 'uneducated guess' which we call a hypothesis. And then we search the internet for information which supports that. We acknowledge challenges but dismiss them. We may even limit observation to ensure that our case studies don't blow up our hypothesis.

So, when I observe differing cultures learning differently, I wonder,  and I try not to generalize. When I see a blind person reading Braille or anyone reading an audiobook, or even (to quote from a Twitter argument with Buck), a telegrapher in an old film reading Morse Code, I wonder about what those things - cognitively - have in common and what separates them. If I see Canadian or European school policy I look at the entirety of the difference, and I, yes, it's the leftist in me, look at who seems to be "winning" and "losing" in the resultant society. I can even look at French laïcité and be both strongly supportive of the goal and deeply troubled by the personal freedom implications.

Now Stuart Buck's interests ("historical" as he says) and mine may not seem to clash. We're both, I suspect, historians by nature, and deeply political historians (is there any other kind?). But I'm a post-modernist and he is not. I doubt truths and he hunts for truths. Still, the biggest separation is that I never begin research with a hypothesis. I am a Grounded Theory guy ("Rather than beginning by researching and developing a hypothesis, the first step is data collection, through a variety of methods."). I begin instead with an interest in a topic, but I refuse to make predictions as to what I will discover.

This does not eliminate bias. Nothing eliminates bias in human learning. But it tends to limit "confirmation bias" because there is nothing to confirm. And it tends to limit confirmation bias because Grounded Theory requires you to define terms. I think of geologist William Smith in Simon Winchester's book The Map That Changed the Worldneeding to begin his work with a definition of "rock." What he was seeing in those coal mines and canal digs of the 1790s couldn't begin to be classified until he determined that definition.

So, I think Buck is wrong about many things. But that's not a problem. Many of us disagree about many things. That's what makes the world an interesting place.

But in why I think he gets it wrong is a cautionary tale for all of us.

- Ira Socol