4 Ways Digital Mental Health Products Run Into Regulatory Risk — Case-Based Analysis
Many digital mental health products do not become exposed because of a single product failure.
They become unstable in how they are understood.
Nothing appears obviously wrong at first. The product functions, the evidence exists, and the messaging is reasonable.
Over time, however, strain begins to appear — in regulatory review, in partner conversations, or internally when teams begin to question what can be said with confidence.
Is it a communication issue and, if so, where does it manifest?
This is often less a product failure than a communication problem—specifically, how the product is described, and where that description begins to break down. In practice, the issue is rarely incorrect claims.
The core problem is misalignment between what the system is, what the content suggests, and what the audience reasonably understands from it.
That misalignment rarely occurs all at once. It develops through incremental adjustments — simplifying language, broadening relevance, and making the product easier to explain across contexts. Each step improves usability of the message while reducing its precision.
The cases below show where that shift becomes visible as consequence.
1. When Trust Language Exceeds System Behavior
Case: BetterHelp
BetterHelp presented itself as a confidential online counseling platform. Users shared sensitive mental health information during signup, including details about depression, medication, and personal history. At multiple points, the platform communicated that this information would remain private and used only to provide counseling services.
According to the Federal Trade Commission, the company disclosed:
- email addresses,
- IP addresses,
- health questionnaire data
to third parties such as Facebook and Snapchat for advertising purposes, without obtaining affirmative express consent before those disclosures.
The issue is not simply that data was shared. It is that the system’s data practices did not align with the level of confidentiality communicated to users. This is where the content breaks.
The platform communicates a contained, confidential environment. The underlying data practices included disclosure of personal and health-related information to advertising platforms. That difference is not visible in a single statement, but it is carried by the overall framing.
Internally, this type of misalignment often appears before enforcement. Legal review becomes more detailed. Statements require additional qualification. Teams begin to reassess wording that previously appeared sufficient.
The outcome was external: regulatory action, a $7.8 million settlement, and required changes to data handling practices.
Trust becomes harder to sustain when data practices do not align with the level of confidentiality communicated to users.
2. When Convenience Narratives Obscure Clinical Weight
Case: Cerebral
Cerebral operated as a direct-to-consumer mental health platform offering therapy and medication management through a subscription model.
According to the U.S. Attorney’s Office, between 2021 and 2022 the company implemented specific prescribing-related practices. These included:
- tracking prescription rates following initial visits
- setting targets such as a 95% initial visit prescription rate
- planning to increase stimulant prescribing for ADHD patients without comorbidities
- monitoring providers by these metrics
- introducing incentives tied to prescribing outcomes
- maintaining insufficient controls to prevent diversion
These documented practices affected how prescribing occurred within the platform, including for controlled substances. This is where the content breaks.
Descriptions of the service emphasize access and integration. The operational structure included measurable mechanisms that influenced prescribing behavior within a regulated clinical domain. The level of control, constraint, and risk associated with those activities was not reflected at the same level in how the system was described.
The issue is not a single statement. It is a mismatch between:
- simplified descriptions of care delivery
- and the operational realities of prescribing within a regulated context
Internally, this often appears as tension between operational metrics and clinical expectations. Questions arise regarding how performance measures relate to clinical judgment and regulatory standards.
The outcome included a Department of Justice investigation, a financial settlement exceeding $3.6 million, and the discontinuation of controlled substance prescribing.
When care delivery includes regulated clinical activities, simplified descriptions can become unstable if they do not reflect the controls, constraints, and risks involved.
3. When Evidence Expands Beyond Its Conditions
Case: Babylon Health
Babylon Health presented results from internal testing suggesting that its AI symptom checker could perform at a level comparable to human doctors in specific evaluation settings.
The underlying study involved a limited number of physicians, exam-style conditions, and did not include validation in real-world clinical environments. Critics raised concerns that the conclusions drawn from this work extended beyond what the methodology could support.
The issue is not the absence of evidence. It is how that evidence is used.
A sequence of translation occurs:
- results from controlled testing are used to support broader claims
- performance in limited conditions is interpreted as general capability
- early-stage validation is associated with real-world applicability.
Each step extends the meaning of the evidence beyond the conditions under which it was produced. This is where the content breaks.
The language reflects a broader level of reliability than the underlying validation supports. The system may demonstrate potential, but the claim no longer aligns with the scope of the evidence.
Internally, this often appears as difficulty aligning technical outputs with public communication. Statements that are accurate within research contexts become harder to support when generalized.
Externally, the result included academic criticism, scrutiny from medical organizations, and regulatory attention related to misleading advertising.
Evidence remains tied to:
- how it was generated
- the context in which it applies
- the conditions it does not yet cover
When evidence is extended beyond its conditions, the claim can remain accurate while no longer being valid at the level it is understood—undermining credibility.
4. When Interaction Changes Without Reframing
Case: Koko
Koko is a peer-support platform connecting users with volunteers who respond to messages during moments of distress. The platform was framed as connecting users with real people rather than providing clinical care.
During an experiment, GPT-3-generated draft responses were introduced, allowing volunteers to use AI-generated content when replying. This affected a large number of messages, and the experiment was widely criticized for lacking informed consent.
The interaction was no longer purely human-mediated. At the same time, the framing of the platform did not change to reflect this shift. This is where the content breaks.
The platform was described as enabling human-to-human support, while responses in the experiment could be partially generated by AI.
The description of the interaction no longer aligned with how responses were produced.
In this case, the central problem was not merely the use of AI, but the lack of alignment between how the interaction was described and how responses were generated.
Internally, this type of change often appears as a product-level experiment that introduces new capabilities without a corresponding update to how the interaction is defined. Questions arise regarding whether user expectations still match the actual experience. The result was immediate public criticism and removal of the feature.
In mental health contexts, interaction definitions shape expectations.
When the structure of the interaction changes, the description must be updated to reflect that change clearly.
What These Cases Show
These cases do not represent isolated or unusual failures. Each case involved a functioning digital health or mental health service, tool, or platform operating within its intended domain.
The issue in each case was misalignment:
- confidentiality language did not match data practices
- simplified care descriptions did not reflect prescribing-related risk
- claims extended beyond the conditions of supporting evidence
- interaction descriptions did not reflect how responses were generated
This is not traditional overstatement. It is misalignment between communication and observable system behavior.
Why This Happens
This pattern does not come from a single decision. It builds gradually, often as part of normal work. Language gets simplified so it can travel across teams and channels. Descriptions are adjusted to make the product easier to explain. What starts as a precise explanation in one context becomes a more general version in another.
At each step, something small is removed. Conditions that once mattered become optional to mention. Limitations feel less necessary. Boundaries stay implied rather than stated. None of this feels risky in isolation. In fact, it usually improves clarity and makes communication faster to use.
Over time, though, the center shifts. The message becomes easier to repeat, but less anchored to how the system actually works.
Inside teams, this doesn’t usually show up as a clear problem. It’s more subtle. People begin to rely on shared understanding instead of defined terms. Different teams describe the same product in slightly different ways. Questions start to take longer to answer, not because the system is unclear, but because the language around it no longer maps cleanly to it.
By the time this becomes visible externally, it rarely feels like something new. It’s more like something that has been present for a while, just not fully examined.
What This Changes in Practice
The task is not simply to make content more cautious, but to keep it aligned with observable system behavior.
Content defines:
- what the product does
- what type of interaction is taking place
- what level of trust is appropriate
- what the system can and cannot do
When content extends beyond what can be directly supported by how the system operates, the effects are rarely immediate. They tend to show up gradually, often in places that feel procedural rather than critical.
Approvals begin to take longer. Statements that once felt straightforward require additional qualifiers. External stakeholders start asking for clarification, not because the message is unclear, but because it no longer maps cleanly to observable behavior. Over time, this turns into increased scrutiny.
None of these signals appear dramatic on their own. But together, they indicate that the communication is carrying more than the system can consistently support. The task, then, is not simply to reduce risk in language. It is to ensure that what is communicated remains aligned with how the system actually operates.
In other words, the objective is to ensure that what is communicated remains consistent with how the system operates across contexts and over time.
Where This Leaves Teams
The companies described above continue to operate. These situations did not end their business. They did, however, require regulatory, structural, and reputational corrections—and those corrections are costly.
Most teams do not set out to create this kind of misalignment. It develops as systems scale and communication becomes easier to reuse.
If the goal is to avoid those costs, the work is to identify where communication is beginning to extend beyond observable system behavior and bring it back into alignment early—before it becomes visible externally.
If useful, explore ways to diagnose where that misalignment is emerging and what to adjust without adding additional complexity.
