Skip to content

Evaluating Therapeutic Interventions: Who is Qualified?

January 9, 2011

Over the past few years, there has been ongoing denigration of academics who have criticized certain novel interventions that either completely lack any published research to support their efficacy and claims or in some cases, the research that does exist is so flawed that it cannot be used to claim efficacy. Rather than respond directly to the critical analysis which has been published in peer reviewed journals, the supporters of these interventions, instead, resort to ad hominem (personal) attacks on the authors of the critiques. One of the most common forms this takes is to attempt to denigrate their credentials, contending that people who are not practicing, licensed clinicians are not qualified to evaluate such interventions. I couldn’t disagree more and here is why.

First of all, most practicing clinicians have very little background in research, especially in evaluating research. This is especially true of those who graduated more than 10 years ago, but even as recently as 2006, a study of MSW and PsyD programs that I discussed in a previous article, indicated that approximately two-thirds of the students had not received a didactic and a practicum in even one evidence-based treatment. Remember, that the majority of practicing mental health professionals are MSWs, other masters level therapists or PsyDs rather than PhDs or MDs (although MD psychiatrists also do not get the best education when it comes to learning research-based practices, according to a recent study reported by Carol Tavris and Elliot Aronson). This is a sad state of affairs, to say the least. Moreover, most of these students have had little more than very basic research courses and since over the past decade, a number of new standards have been put in place for evaluating research, most practicing clinicians, unfortunately, would not know where to begin, when it comes to evaluating interventions. Some of us are working very hard to change this so students are properly trained to evaluate research, but the present reality is what it is.

In his recently published book, What is Mental Illness (an excellent book which I plan to review here), Harvard psychologist Richard J. McNally reported that according to the National Comorbidity Survey Replication (NCSR), a study that conducted psychiatric interviews with more than 9,000 adults throughout the US, found that only one-third of those who were in therapy had received either pharmacological or psychological treatments that had research to demonstrate their efficacy. This is interesting because it means that approximately two-thirds of those in therapy were receiving interventions that lack such support.

Clearly, the majority of practicing clinicians either are not aware of or have decided against practicing interventions that have empirical support and if they are not able or willing to practice such interventions, then how do you think it will be that they could be in any way qualified to evaluate an intervention’s evidence, which takes even a higher degree of training and skill? There are, however a number of people in academia who, although not practicing clinicians, do have the level of coursework and skills necessary to do a thorough evaluation of such interventions. Since the majority of clinicians are failing to deliver the best possible interventions to their clients, then obviously those from the outside who may not be clinicians need to step in and provide such an evaluation. Whether they have ever seen a client or not is irrelevant, since what is being evaluated here is the research evidence, or lack thereof to support any particular form of therapy.

Reversal of the burden of proof is common. Critics do not have any obligation to conduct empirical studies. Rather, the burden of proof is on those who are making the claims that a particular form of therapy is effective. As can be readily observed, there are many clinicians who are practicing and charging very high fees for such interventions and yet have not gone to the trouble to have randomized clinical trials, conducted by people who have no vested interest in such interventions. I find it highly ironic that Jean Mercer’s detractors are criticizing her for not conducting empirical studies of the interventions she criticizes. Dr. Mercer has no obligation or onus to conduct such studies The onus is on those making the claims.

Additionally, let’s put the myth being circulated to rest once and for all that scholarly criticism is mere “polemics”. It is not. Critical analysis is a valid, legitimate part of scholarly work and more and more journals are beginning to recognize the need for such critical analysis. For example, Jean Mercer and I published an article that presented a detailed critique of the existing research on Holding Therapy and Dyadic Developmental Psychotherapy, identifying exactly what the flaws were in the research. These are not mere “polemical” articles, but rather, specific critiques. The same holds true for the systematic review I just published in the peer reviewed journal, The Scientific Review of Mental Health Practice, where I conducted of the literature on reactive attachment disorder and adoption.

While I am not suggesting that people ignore anything said by a clinician, given that so many are using methods that are not supported by well-designed research, it would behoove people to ask very specific questions about what research exists to support the efficacy of the interventions being offered and if the answers you are being given are vague, do your own searches and pay attention to critical reviews published in peer reviewed journals by people who are well trained and qualified to evaluate such evidence. If a clinician is denigrating people who conduct such reviews, that is a very bad sign, in my opinion. Clinicians who deliver evidence-based interventions have nothing to fear from such reviews and have no problem submitting their work to critics.

A recent prominent example is the British Medical Journal article that further exposed what has now turned out to be research on vaccines, long known to be highly flawed and retracted last year, that has been declared fraudulent. Such research, over the years, has frightened many parents away from having their children vaccinated after being frightened away by such misinformation and poor research, that was unfortunately further publicized by celebrities and one anti-vaccine activist, Barbara Loe Fisher, even attempted unsuccessfully to sue Dr. Paul Offit who was a proponent of vaccination, taking words “she lies” that the Eastern District Court of Virginia determined were taken out of context and as the language of a heated controversy are part of our constitutional rights to free speech. In other words, anti-vaccination proponents, it seems, resorted to whatever tactics they could to attempt to silence their critics, but ultimately they failed miserably, even though unfortunately, there are true believers that remain.

Does it matter if the people who exposed this fraud such as investigative reporter Brian Deer were themselves not doctors who saw patients? Of course not. The issue here is the research itself and one does not need to be a doctor, to recognize and expose its serious flaws. Here is a summary of Brian Deer’s investigation and conclusions.

What does all this mean for health and mental health consumers? Again, I repeat, I’m not suggesting that clinicians should be ignored, but what I would recommend is looking at a wide variety of sources of information and if such clinicians are actively discouraging consumers from doing this by vigorous attempts at denigrating critics and only going to people for feedback that the clinician or that person’s supporters recommend, that, in my opinion would be someone I would not want to hire as a therapist for myself or anyone in my family. If someone was actively trashing a critic and telling me not to listen to that person, that would be the very first person I would want to contact and at least listen to what they have to say — they may be right or wrong, but when someone is actively trashing critics, that is a very bad sign.

As always, this article contains my opinions on what are very heated, controversial issues, opinions that I cherish and never take for granted, my constitutional right to express.

  1. Would you please provide a citation for the NCS-R study you mention (interviews with 9,000)? Thank you.

    • This was cited in McNally’s book. The reference he gave for it is:
      P.S. Wang, M. Lane, M. Olfson, H. A. Pincus, K. B. Wells and B. C. Kessler. “Twelve-Month Use of Mental Health Services in the United States. Results from the National Comorbidity Survey Replication,” Archives of General Psychiatry, 62 (2005), 629-640.

      59% of those with disorders did not seek treatment at all and of those who did seek help, only one-third received treatments with empirical support.

  2. anyachaika permalink

    I agree your view that advocates of practices such as holding therapy have a duty to respond to critical analysis. There is very little academic debate on this subject in the UK, apart from the recent article in the BJSW, and almost no intelligent public debate. My impression is that shedding more light on these things is the most effective way to prevent these services from being commissioned by Local Authority managers. I’ve started a blog to try to start this process at

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: