False Positives and the Dangers of Untested Models
In the world of pop psychology, untested mental health models, often replete with checklists abound. These can be tempting and compelling when people are looking for easy answers to complex problems and so they fulfill a need, hence their popularity. One example of such a model is Steve Hassan’s BITE model. BITE stands for behavior, information, thought and emotional control and is a way offered by Hassan to evaluate whether a group in question should be considered a cult. Go here to read Steve Hassan’s own description of his model. While the model does make a certain amount of intuitive sense and seems to fit a number of groups that most people would consider cults, there are dangers, especially since this model has never been tested through systematic research and yet is routinely used to assess various groups that people have questions about.
One such controversy has been whether the Church of Jesus Christ of Latter Day Saints, the Mormon religion, is a cult. Some people say yes, others say no. Several ex-Mormons who have left the religion have applied Hassan’s BITE model, which they believe proves that it is indeed a cult. I say not so fast. If a model hasn’t been tested, one of the dangers is that while it might correctly identify groups as cults, it may also have false positives. That is, groups that are not cults might also be made to fit the model, especially when applied by people who have motivation to want to make it fit.
When it comes to assessment, there are two main functions: 1) to correctly identify what is being assessed for and 2) to identify only that and not identify false positives. A well known example of the latter is the famous symptom checklists from self help books published in the 1990s that supposedly identified people who had been sexually abused in childhood and repressed the memory. The problem is, these symptom checklists were so broad, they could identify a number of people who had never been abused. The same holds true, as I wrote about in an earlier blog, for symptom checklists for people who have been in cults that supposedly indicate that they have issues that need to be addressed, but can just as easily be descriptive of problems that have nothing whatsoever to do with cults.
Hassan’s BITE model may suffer from this same problem and unfortunately, some of the ex-Mormons who believe they have successfully applied it to the Mormon religion have not grasped this point. For example, ex-Mormon Luna Flesher wrote an essay attempting to refute people who have argued against the Mormon religion being a cult, by maintaining that when people point out that the model fits other major religions, they are invoking the tu quoque fallacy. Not exactly, Ms. Flesher. This fallacy means “you too”. When someone is criticized, the respondent says “you too” suffer from the same thing. It is the proverbial pot calling the kettle black. For example, if someone who frequently lies calls someone else a liar, the accused person might respond “you lie too”. This is a fallacy because it does not in any way lessen the charge that the accused person lied. An example of a tu quoque fallacy in the area of cults would be a Scientologist calling a Moonie a cultist and the Moonie responding, “you’re a cult too.” Here, we are dealing two groups that most people knowledgeable about cults have actually agreed are cults and hence, the Moonie saying “you too” to the Scientologist is invoking the tu quoque fallacy.
However, pointing out that a model lacks sufficient specificity and identifies false positives, is not the tu quoque fallacy. It is a valid criticism, since valid assessments must only identify true cases of what is being measured, not false positives. One way to determine this is to take a case that would clearly not fit what is being assessed for and see if the assessment comes up positive for that case. For example, if someone can make the BITE model fit a mainstream religion that most people knowledgeable about cults would agree is not a cult, then we have a problem with Steve Hassan’s model. For example, can Steve Hassan’s model be made to fit the Catholic church? A graduate school program? A born-again Christian group? A major corporation? While all these types of organizations may, at times, exhibit cult-like features, that would hold true for just about any large organization. A good test for whether a group is a cult, would need to go beyond that and identify only groups that are destructive cults, and eliminate those that are not. The problem comes in when we are trying to use the BITE model to identify a group that experts disagree over, namely the Mormon religion. Some people consider it a cult, whereas others do not (in my own opinion, it is not). A good assessment should be able to resolve the question. Think of it this way: would you want to take a medical test for cancer that would result in a decision to undergo chemotherapy that successfully identified cancer, but also identified people with benign tumors?
The problem is that since Steve Hassan’s BITE model has never been formally tested by good research, I wouldn’t want to rely on it to make what could result in life-changing decisions. One way to test the validity of the BITE model would be to take a list of groups that clearly, according to most cult experts, clearly fall into the category of “cult” and then take another list of groups that clearly most experts would consider “not cults” and then recruit participants to use the BITE model to rate the groups from both the “cult” list and the “not cult” list. If the groups on the “cult” list come up positive and the groups from the “not cult” list come up mostly negative, that would provide good evidence for the validity of the BITE model, both in terms of correctly identifying what it was intended to and not identifying what it was not intended to. If, however, the model successfully identified the “cults” but also identified the “not cults” there would be a major problem with the BITE model. It would mean that if someone was motivated not to like a particular group, they could apply the BITE model to it and declare it a cult. This may be what is happening, in some cases and since the model has not been systematically tested in this manner, preferably with the results published in a reputable peer reviewed journal, preferably one that does not only include anti-cult insiders, my own advice would be to apply this model with caution and do not use it to make life-changing decisions.
In a recent interview Steve Hassan gave to a Las Vegas Nevada ratio station, KNPR, where he commented on a new Unification Church (Moonie) center that was being set up in Las Vegas, the interviewer asked him how the Moonies were any different from the Mormon missionaries. Much to the interviewer’s surprise, Hassan stated that he also had a problem with Mormon missionaries. Since the topic at hand was the UC, they did not pursue it further, but on Hassan’s Freedom of Mind website, there are essays from ex-Mormons on how they believe the Mormon religion and the two year missionary program fits Hassan’s BITE model and that is his basis. The ways in which it does not fit what most people would consider to be a cult, are explained away and the way it fits are focused upon. This is what is known as confirmation bias. Looking at confirming evidence and explaining away disconfirming evidence. For example, Mormon missionaries are very up front from the very beginning that they are, in fact, Mormons. This has not been the case on numerous occasions, with Moonies. The Mormon religion does not engage in shunning of defectors. If a family member leaves the religion, the family still remains in contact with that person and is not pressured or required to cut off contact. These are just a few key differences. Some of the ways Mormonism fits the BITE model are ways in which many mainstream religions also fit the model. For example, many charismatic born-again Christians who are not considered cultists, are nevertheless strongly encouraged to proselytise to bring in new members and consider it their Biblical duty to do so. Born again Christians often tell of dramatic, life-altering conversion experiences that may well lead them to severing ties with a previous way of life. This does not necessarily make it a cult. A key distinction is whether the person made a fully informed choice. Mormon converts are fully informed about the religion’s beliefs before being allowed to be baptized, including the fact that there are temple rituals that are only disclosed to people who are married in the temple (in other words, yes, they are “secret” but the person knows, going in, that there are such secrets). This is very different from what Steve Hassan has described in his own writings about his experience in the Moonies, where he had no idea that their form of arranged marriage even existed, at the time of his conversion.
Just in case some people are asking, where’s the harm in this, take a moment and think about it. Just as a false positive on a cancer test could result in unnecessary unpleasant treatment that would not be worth the risks if one did not have cancer, the treatment prescribed by anti-cultists could also have undesirable effects, including unnecessary expense, stress and interference with people who have made unconventional, but fully informed choices in their lives. While this may be good for the business of “cult mind control” specialists, I’m not so sure about the consumers. Hence, I have to caution, buyer beware.
The BITE model is just one example. There are all kinds of other models and checklists that one can find on the internet that, while seductive and even fun to do, lack adequate testing for their validity. Another popular example is the checklist for having grown up with a narcissistic parent. Keep this in mind, next time you fill one out.
