In the last few years, an abundance of consumer-facing artificial intelligence (AI)-powered mental health tools have appeared on the market. The prevailing narrative is that generative AI can help close the gap between profuse mental health needs and insufficient care capacity. Yet there are already far too many cases of its use leading to tragic user outcomes. Even today’s most sophisticated generative AI cannot be fully relied upon, as it lacks human nuance and discernment.
The most important role AI may play in mental health is not in replacing human interaction, but in helping people connect better. Rather than supplanting therapists or support systems, AI’s greatest value may lie in recognizing patterns humans can easily miss, making early detection and intervention one of its most powerful contributions to digital mental health.
Why Detection Is AI’s Greatest Strength in Digital Mental Health
Much of the casual conversation around AI in mental health has focused on whether machines can replicate human empathy. With iterative improvements, it most certainly can, but the more important question is should it? From research examining loneliness and mortality, we know that disintermediating humans from human connection is detrimental and that lonely, disconnected humans face increased risk of cardiovascular disease, depression, and even earlier death. Questioning whether the future will look like a sci-fi film where humans’ best friends are machines takes away from the real world cases where humans’ reliance upon technology fails to fill the loneliness gap.
Instead, we could focus on AI’s strength, which lies in its ability to continuously work without delay and its rapid pattern recognition for continually flagging risks in real time. This, in turn, could support human detection of safety issues as well as support earlier intervention.
Suicidality is readily detectable, and there are no excuses in today’s day and age for leaving safety gaps. AI may be the most powerful safety guard digital mental health professionals can utilize to ensure the safest authentic human care.
What the Data Reveals
The Journal of Clinical Medicine recently published a study exploring the frequency of suicidal ideation and the response time of scalable AI-led crisis detection tools.
The study investigated 169,000+ synchronous peer-to-peer chat transcripts from a digital peer support live chat service with real-time human moderation. Suicidal ideation (SI) was confirmed in about 3.2% of chats, showing the regularity of these high-risk discussions.
Most notably, the study discovered that a specially trained risk detection AI model predicted and flagged suicidal users faster than the trained human moderators did, 78% of the time, with over 90% agreement between the AI and the moderators.
Once user messages were flagged by AI, the moderators responded within seconds, enacting a quick protocol to assess severity of risk on the spectrum of passive to active suicidality, and routing users in active states of SI to appropriate higher-acuity crisis care as needed.
The implications are clear: 1) we cannot ignore that in any emotion-focused conversation, risk of suicidality may be disclosed, and 2) a hybrid model of both AI and human detection may close the mental health safety gap that AI presents when attempting human emotional support alone.
The Benefit for Healthcare Leaders
For leaders in all facets of health care and delivery, the importance of safe and ethical AI use is ever growing. As demand for generative AI capabilities on all health topics grows, it is paramount that safety measures are embedded into all machine-led interactions, especially for those covering emotion-sensitive topics. Any AI conversational touchpoint should include safeguards and capabilities to effectively and rapidly triage at-risk users to high-acuity human care.
Through integration of dedicated, reliable, and proven crisis monitoring AI models, and better practices that combine AI and human oversight into a well-functioning hybrid system of care, healthcare leaders can in turn protect their businesses and teams from user harm and liability risks.
As AI progresses in its goals of mirroring human thought and behavior, it is imperative to remember that progress is not in recreating a human-mimicking machine, but, more importantly, in how the machine can support real humans. Allowing us to capitalize on what AI does best, while letting real people use the power of genuine human empathy to do what mental health professionals do best: connect, serve, and support, one safely supported conversation at a time.
Helena Plater-Zyberk is the Founder & CEO of Supportiv, the AI-driven on-demand peer-to-peer support service that serves large employers, EAPs, health plans, hospitals, Medicare, and Medicaid and has helped over 3 million people cope with, heal from, and problem-solve struggles like stress, burnout, loneliness, parenting/caregiving, anxiety, and depression. Supportiv has been proven in peer-reviewed research to reduce the cost of mental health care and deliver clinical-grade outcomes. She previously served as CEO of SimpleTherapy, an at-home physical therapy service, and has operated business units for global corporations Scholastic and Condé Nast. Helena holds an MBA from Columbia University.
