\n<\/p>\n
Does generative AI overlook rare mental health conditions or is it doing the right thing by avoiding low chance aspects?<\/span><\/p>\n getty<\/small><\/div>\n<\/div>\n<\/figure>\n In today\u2019s column, I examine the key challenges associated with generative AI and large language models (LLMs) in detecting rare mental health conditions. <\/p>\n Most of the everyday usage of popular LLMs to get mental health advice usually involves people with relatively common cognitive issues, such as depression, anxiety, and the like. But what about situations when a person has something much less common?<\/p>\n A rare mental health condition can be like a needle in a haystack. The odds of discerning that someone potentially has a rare mental health condition can be extremely difficult. The same difficulty confronts human therapists and psychiatrists. They too can falter in not considering a rare condition since it isn\u2019t on top of mind, or they set aside the chances of the condition arising because they know it is extraordinarily rare.<\/p>\n To illustrate this conundrum when it comes to AI, I will focus on a rare mental health disorder known as intermittent explosive disorder (IED) to showcase the challenges underlying these circumstances. Human therapists and generative AI can readily miss the mark when it comes to suitably determining that IED might be present.<\/p>\n Let\u2019s talk about it.<\/p>\n This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here). <\/p>\n As a quick background, I\u2019ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For an extensive listing of my well over one hundred analyses and postings, see the link here and the link here.<\/p>\n There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance on an episode of CBS\u2019s 60 Minutes<\/em>, see the link here.<\/p>\n<\/section>\n I\u2019d like to set the stage on how generative AI and large language models (LLMs) are typically used in an ad hoc way for mental health guidance. Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of which dip into mental health aspects, see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.<\/p>\n This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24\/7 basis. <\/p>\n There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines in August of this year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement. <\/p>\n Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm. For my follow-on analysis of details about the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. As noted, I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards.<\/p>\n Today\u2019s generic LLMs, such as ChatGPT, Claude, Gemini, Grok, and others, are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to presumably attain similar qualities, but they are still primarily in the development and testing stages. See my coverage at the link here.<\/p>\n<\/section>\n Analyses of how people use generic LLMs for mental health tend to suggest that the bulk of the chats are typically about relatively common psychological issues. For example, people often ask AI how to cope with depression. Another frequent aspect is seeking insights into how to deal with anxiousness and anxieties. These are common conditions and represent the mainstay of such AI chats.<\/p>\n Of course, not everyone necessarily has a common consideration at hand. <\/p>\n What about people who have rarer mental health conditions? The odds are that the LLM will fail to detect the condition. Furthermore, the AI is likely to try to squeeze the circumstance into a common condition since that\u2019s what is more likely to be the case. Any symptoms that point toward depression, anxiety, and other common possibilities will nudge the AI computationally toward assuming a common condition is the culprit.<\/p>\n The AI will ostensibly attempt to place a square peg in a round hole. This is an instance of a false positive. The AI falsely calculates that a person must have a common condition due to the expected frequency of common conditions. Not detecting the rare condition is an instance of a false negative, namely, the AI passed over identifying the actual condition. Initial data training of the AI is bound to have established this kind of pattern. The LLM scanned lots of data about common conditions, and seldom scanned data about uncommon mental health conditions.<\/p>\n Human therapists are somewhat in a similar boat in the sense that they are likely to receive training on more common conditions and less training on uncommon conditions. In addition, in their day-to-day therapeutic practice, they are likely to see roughly the same common conditions repeatedly. They seldom have clients who have rare mental health conditions. All in all, a rare mental health condition is not something that one generally anticipates encountering.<\/p>\n<\/section>\n To illustrate the difficulty of discerning rare mental health conditions, consider the psychological disorder known as Intermittent Explosive Disorder (IED). <\/p>\n This disorder has a decidedly low base rate. Its symptoms often overlap with bipolar disorder and other possibly coexisting conditions. People with this disorder can try to hide the symptoms from a therapist or aim to rationalize their behavior to mask the condition. A kind of comorbidity fog can be created that leads to vastly underdiagnosing the condition.<\/p>\n A research article entitled \u201cPsychiatric, Neurological, and Somatic Comorbidities in Intermittent Explosive Disorder\u201d by Yanli Zhang-James, John Paliakkara, Joshua Schaeffer, Joseph Strayhorn, Stephen V. Faraone, JAMA Psychiatry<\/em>, January 22, 2025, made these salient points (excerpts):<\/p>\n Overall, this condition is an understudied psychiatric disorder and would be unlikely as a candidate condition on the mind of a generalist therapist. The key is to be mindful of symptoms, including impulsive aggression, inadequate regulation of emotional control, disproportionate reactions and outbursts, and is usually accompanied by adverse interpersonal and societal consequences.<\/p>\n<\/section>\n I decided to log into a popular generative AI and pretended to be someone with intermittent explosive disorder. I didn\u2019t come right out and claim to have the condition. Instead, I gave clues that I hoped would drive the AI in that direction. After repeated attempts to give sufficient details about my IED-related behavior, the AI kept coming back to common conditions and said I might be experiencing ADHD, PTSD, and so on. <\/p>\n The matter of intermittent explosive disorder never came up by the AI. I tried very hard to prod the AI in that direction by laying out a litany of stories about having outbursts, impulsive aggression, etc. No dice.<\/p>\n One upbeat argument is that it is perhaps best that AI didn\u2019t land on that rare condition. If it had, this might be a worrisome sign. Perhaps the LLM is too easy in choosing rare conditions. We don\u2019t want AI to be widely telling people that they might have a rare mental health condition. In that sense, it is nearly reassuring that the LLM wasn\u2019t going to be a pushover and fall into the detection of a rare condition.<\/p>\n Another factor is that a human therapist would conventionally wait to make such a diagnosis until they believed there had been plenty of indicators associated with the condition. We presumably want AI to act similarly. A human therapist would undoubtedly collect a lengthy behavioral history, seek to identify collateral reports, and aim to see a pattern consistency over time and across multiple therapy sessions.<\/p>\n AI also ought not to rush to judgment.<\/p>\n<\/section>\n One possibility is that the popular generative AI didn\u2019t have any data training associated with intermittent explosive disorder. If there had been no scanned documents during initial training that mentioned this disorder, there wouldn\u2019t be any patterns established within the AI to then tap into when the moment arose. <\/p>\n No data, no detection.<\/p>\n I went ahead and showed my cards by asking the AI whether it had any contextual aspects about the disorder. It did. So, the disorder was already previously data trained on. The data training could be in-depth, or it could be shallow. The point being that just because AI has encountered a particular topic doesn\u2019t mean that there is much within the AI about it.<\/p>\n I started a new conversation with the AI. Once again, I began by pretending to have the disorder, though I didn\u2019t say that I have it, nor did I mention it at all.<\/p>\n Here\u2019s what happened.<\/p>\n Aha, notice that the AI now brings up the intermittent explosive disorder. Why? Because I had just carried on regarding the disorder, and the matter was now within a recent context. <\/p>\n This is good news and bad news. The good news is that the AI has now increased the chances of my having the disorder. It is no longer at the bottom of the list. The bad news is that the AI might become preoccupied with my potentially having the disorder. As such, anything else that I say could be interpreted as a sure sign that I do have the rare disorder.<\/p>\n Look at what occurred next.<\/p>\n When I asked the AI how it landed on IED, the response indicated these core features seemed to be present: (1) Recurrent and sudden anger outbursts, (2) disproportionate reaction to the triggering, (3) poor impulse control, (4) brief episodes, (5) remorse, shame or confusion afterward, (6) no apparent explanation via use of substances.<\/p>\n<\/section>\n It is incontrovertible that we are now amid a grandiose worldwide experiment when it comes to societal mental health. The experiment is that AI is being made available nationally and globally, which is either overtly or insidiously acting to provide mental health guidance of one kind or another. Doing so either at no cost or at a minimal cost. It is available anywhere and at any time, 24\/7. We are all the guinea pigs in this wanton experiment.<\/p>\n The reason this is especially tough to consider is that AI has a dual-use effect. Just as AI can be detrimental to mental health, it can also be a huge bolstering force for mental health. A delicate tradeoff must be mindfully managed. Prevent or mitigate the downsides, and meanwhile make the upsides as widely and readily available as possible.<\/p>\n Benjamin Franklin famously made this remark: \u201cRemember not only to say the right thing in the right place, but far more difficult still, to leave unsaid the wrong thing at the tempting moment.\u201d The same applies to AI that discusses mental health. We want the AI to say the right thing in the right place, and leave unsaid the wrong thing. It\u2019s a high bar and one that we must aim to attain.<\/p>\n<\/section>\n<\/div>\nAI And Mental Health<\/h2>\n
Background On AI For Mental Health<\/h2>\n
Common Versus Uncommon Mental Health Issues<\/h2>\n
Illustrated Via Intermittent Explosive Disorder<\/h2>\n
\n
Example Dialogue And Pretense<\/h2>\n
Tipping AI Into The Rare Conditions<\/h2>\n
\n
\n
The World We Are In<\/h2>\n