Bethany Brand, Conrad Murray, Debbie Nathan, Dissociation, Dissociative Identity Disorder, International Society for the Study of Dissociation, ISSD, Judith Peterson, meta-analysis, Michael Jackson Death, Monica Pignotti, Multiple Personalities, NPR, Paul McHugh, propofol, Robyn Dawes, Sybil, Sybil Exposed
NPR Again on Multiple Personalities: This time on Sybil and Inflated Claims about DID Therapy
Previously, I wrote about an NPR debate on Science Friday on multiple personalities. This past week on Science Friday, there was yet another debate, this one between Professor of Psychiatry at Johns Hopkins Paul McHugh and Associate Professor at Psychology at Towson University, Bethany Brand. The main guest was Debbie Nathan, author of the recent book, Sybil Exposed.
Once again, I heard what are, in my opinion, overinflated claims from the pro-DID side. I checked out the actual evidence claimed by Dr. Brand, by obtaining a full copy of an actual meta-analysis she cited, which was one of her primary sources for the claims she was making of solid science behind DID and its treatment. I say, not so fast, Dr. Brand. She referred to a meta-analysis which she believes supports the efficacy of DID therapy, which she portrayed at empirically supported. I looked up and obtained a full copy of the actual meta-analysis (Dr. Brand was the first author) and found it to be anything but. Although Dr. Brand was portraying the evidence as scientifically strong, in her peer reviewed publication, she was much more circumspect, stating the serious flaws and limitations in the studies reviewed. There seems to be a very big difference between how Dr. Brand portrayed the evidence to the media and how she portrayed it in her journal article. This is unsurprising, since reputable peer reviewed journals force their authors to state study limitations. The following is a commentary I wrote about this program, that I have posted elsewhere. As usual, the following is my opinion.
I have now obtained a copy of the meta-analysis which Dr. Brand was first author and can be more specific. It appears that in her media interview,she has grossly overstated the evidence for the 8 uncontrolled studies, some with quite high dropout rates, that involved a vaguely described array of treatments, with no protocol clearly and transparently described (as current reporting guidelines call for) with insufficient fidelity checks (which would be impossible without such a clear and transparent description). Moreover, it is very limited what one can do statistically with a meta-analysis involving so few studies.
Since the dropouts in the individual studies included were not properly evaluated, there is no way to know whether the treatments did harm. One study had an 80% dropout rate and another, a 50% dropout rate. There is no way these studies reviewed would meet any criteria that I am aware of (e.g. the APA criteria) for the “empirically supported treatments” she is claiming in the NPR interview. The APA criteria for empirically supported treatments have been criticized for being too lenient, but this doesn’t even appear to meet those criteria. In the actual peer reviewed article, she was more transparent about the serious limitations of those studies. Here is a link to the abstract which includes the full citation:
where she wrote:
There are significant methodological limitations in the current DD treatment outcome literature that reduce internal and external validity including regression towards the mean, limited sample sizes, and nonrandomized research designs. Implications for future research and treatment planning for patients suffering from DD are discussed.
In her conclusions, in the body of the paper, she elaborated even further (this appears to be very different from what she represented in her media appearance):
Although it is not surprising that the field lacks RCTs of DD, this review highlights serious methodological flaws in existing research and shows that we must do a better job in our efforts to gather evidence for effective treatment of DD. Given the limitations and challenges of conducting RCTs with DD patients, as well as many other complex disorders with high comorbidity, we must consider alternative approaches to amass empirical support for effective treatment of DD. (p. 653)
She also included a detailed table that describes the serious limitations of the studies reviewed including very high drop out rates, no control group and a number of others. How she got from her honest statement of “we must do a better job” to what frankly I see as hype in her NPR appearance, I do not know. There is nothing, even in that multi-site study that would permit her to do that with any credibility.
It appears that based on what she herself wrote in her actual publication, Dr. Brand has grossly overstated the evidence in support of treatment for DID. She gave none of the caveats in her media appearance that she gave in the actual publication. Dr. McHugh did raise appropriate substantive rebuttals, to the extent that the very limited time permitted. It is not appropriate to reverse the burden of proof and demand that Dr. McHugh do studies to show DID therapy is not effective. That burden remains on Dr. Brand and other proponents and it appears that she is far from meeting it.
As for the study involving a population of death row inmates, this cannot be generalized to other patients, as Dr. McHugh correctly pointed out. It is unsurprising that they had history of trauma, but that doesn’t mean that their DID diagnoses were valid. Also, in response to one of the commenters on the website for NPR, the brain scan studies cannot be taken as proof that DID is a brain disorder, as it leaves open a number of other possibilities. Perhaps the commenter was unaware that small hippocampus has also been found in people with PTSD, not just DID and even those findings were called into question with later twin studies, showing the twin who was not exposed to a trauma also had a small hippocampus, so it appears that the small hippocampus came first, not the trauma.
As for the multi-site study, has this been published in a peer reviewed journal? Presentation at a conference consisting of other proponents is not sufficient for such a study to be properly evaluated to see if it conforms to current reporting guidelines (which would include a transparently described treatment protocol, appropriate fidelity checks, dealing with dropouts, etc.). If anything has been published in a journal independent of the ISSD (which would obviously be biased in favor of proponents, most of whom make their living practicing this therapy – Robyn Dawes wrote about this phenomenon of self-proclaimed “experts” getting together and forming a closed circle that include critics in his book, House of Cards). The distinction he makes is between experts who base their expertise on actual scientific evidence and those that claim expertise without such a basis. I would like to see the exact reference, but based on the way it is described, it sounds like an improvement over earlier studies but still falls short of meeting any criteria for an empirically supported treatment, as she seems to be claiming. The science behind this appears to be far from solid, as she implied in her recent media appearance.
For those unfamiliar with Dr. McHugh’s credentials, he is listed in the 2011 director for Johns Hopkins as a Professor:
although the fact of someone’s retirement should in no way diminish their qualifications and Dr. McHugh’s are quire impressive. I note this to provide some balance to previous comments where Dr. Brand’s credentials were presented as impressive and Dr. McHugh was presented (not on NPR but by a DID proponent discussing this debate) merely as “retired”.
Dr. Brand also referred to the ISSD, as if it were a credible professional organization. That, again, is a matter of opinion. I have trouble finding an organization such as ISSD credible, when it was founded by Dr. Bennet Braun who in the 1990s had a notorious court case where his patient was awarded $10 million + damages and his license was suspended for some pretty outrageous treatment. I also, after reading the therapy transcripts of the work of Judith Peterson, have trouble with an organization who would defend her practices and see her, not her client, as the victim.
The way the scant evidence for DID therapy was portrayed by Dr. Brand on the NPR program was, in my opinion, a classic example of what Eileen Gambrill has recently written about where research reporting is reported in such a way that it becomes an advertisement and propaganda, rather than a transparent scholarly report. Although in the actual publication of the meta-analysis, Dr. Brand at least did state the limitations of the studies reviewed, in other places, she gave vague descriptions of what the DID therapy treatment actually consisted of. Based on her descriptions, it appeared to be a conglomeration of a number of different approaches, which would make replication and fidelity checks impossible.
Fortunately, today there are standards for reporting outcome studies, which include clear, transparent descriptions of what was done in the interventions and rigorous fidelity checks. There are also guidelines for meta-analysis that are widely accepted in the mental health professions. Just putting together a bunch of studies where what was done is not at all clear, does not cut it, by today’s standards. Again, what we appear to have is a large gap between how studies are presented in the media by proponents and the actual study, as it appears in print in the journal.
We also saw some of this at the Conrad Murray trial, where the defense attempted to put, as evidence, a small, highly experimental study on the use of propofol for insomnia. Although the defense even tried to claim it was a “cure” this was quickly shot down by the prosecution and their expert witnesses, who, I believe, conclusively demonstrated that it was an extreme “unconscionable” deviation from standard care to have used the powerful surgical anesthetic propofol on Michael Jackson, night after night, without the proper safety equipment, for his insomnia. Thankfully, the courts now have the Daubert standard, which means evidence presented must be scientific and this has also had an impact on mental health cases, such as the recent victory for the prosecution in a case that tried to use the DID defense.
Programs such as this recent NPR debate point to why we need to continue to call people out, who over-inflate claims of scientific evidence for potentially harmful and questionable therapies.
See part 2, which is the next posting on this blog, for a continuation of my discussion of the NPR program and claims about DID therapy.