Beyond Digital Band-Aids: A Youth-Centered Framework for AI Mental Health Governance

Original Proposal by Nikita Vijay

Executive Summary

During my freshman year of high school, I led a team developing an AI chatbot to detect early signs of depression in students through linguistic drift analysis and established clinical frameworks like DSM-5 and PHQ-9. Recognized at the Science & Engineering Fair of Houston, the project showed promise but also raised policy concerns. When we presented it to our school counselor, their first questions weren’t about our technology. They were about privacy: Who would access the data? Where would it be stored? How long would conversations be retained?

This essay outlines how the U.S. should prepare for AI’s societal impacts on equity by protecting vulnerable youth and ensuring fair access to safe mental health innovations.

That exchange revealed a troubling reality: most AI mental health platforms lack oversight or transparency. Even well-intentioned AI can cross therapeutic boundaries without proper guardrails. Without stronger regulations, private teen conversations risk being used to train future models, a practice that threatens users’ rights, privacy, and trust. Our takeaway was clear: technology alone is not enough; privacy and accountability must come first.

Over 70% of U.S. teens use AI chatbots for companionship, and about a third for relationships or social interaction (Common Sense Media, 2024). Without guardrails, these tools can miss crises or exploit vulnerability instead of oƯering real support.

Drawing on my experience building depression-screening technology with built-in safeguards like crisis-escalation protocols, I propose the Youth Digital Wellbeing Protection Act to make these tools safe by design. The Act creates three core protections:

1. Mandatory pre-use disclosures and verifiable parental consent for minors.

2. Registration for mental health apps, requiring model cards, data transparency, and independent safety reviews.

3. Strong data rights, banning training on minors’ conversations without consent, setting strict retention limits, and requiring annual third-party audits.

To enforce these rules, an independent oversight board within HHS would set standards, apply penalties, and run sandboxes that let responsible innovators test safely.

I. The Crisis Hidden in Plain Sight

The Scale of Youth Mental Health Crisis

The mental health crisis among American teenagers has reached alarming levels. According to the CDC’s 2024 Youth Risk Behavior Survey, 40% of high schoolers report persistent sadness or hopelessness, up from 30% in 2013. The American School Counselor Association (ASCA) acknowledges the growing demand for mental health support and the challenges of meeting these needs. With student-to-counselor ratios at 1:385, many teens turn to free, 24/7 platforms like Replika and ChatGPT, which have become de facto counselors for millions.

The Commodification of Vulnerability

What appears as democratized mental healthcare may represent the quiet monetization of teen emotional distress. When teenagers share their deepest fears with AI chatbots, they rarely understand their vulnerability is being harvested. Current platforms exploit regulatory gaps in HIPAA, COPPA, and international data protection laws, particularly when serving minors.

II. Regulatory Gaps and International Failures

United States: The Wild West

In the U.S., many AI mental health chatbots avoid FDA oversight by framing themselves as wellness tools rather than Software as a Medical Device (SaMD). HIPAA rarely applies unless tied to a covered provider, and COPPA’s child privacy rules are often bypassed by weak age checks. The American Psychological Association warns some chatbots falsely present as trained therapists (APA, 2025), while Mozilla found Replika failed all privacy and security tests, leaving minors exposed (Mozilla Foundation, 2023). State laws such as California’s Age-Appropriate Design Code (2022), Massachusetts’s AI mental health approval bill (H1974), and California’s healthcare AI oversight mandates (AB-3030, SB 1120) attempt to fill gaps but diƯer widely in scope and enforcement.

European Union: Incomplete Protection

The EU AI Act, in force since August 2024, classifies high-risk AI and bans systems that exploit age-based vulnerabilities. While it doesn’t explicitly target mental health chatbots for minors, its prohibitions on manipulative systems and emotion recognition in educational settings create relevant safeguards. The GDPR adds stronger privacy rules, including parental consent for processing children’s data, but enforcement is inconsistent. In one case, Italy’s data protection authority fined Replika €5 million (European Data Protection Board, 2025) for having no age verification and no way to block underage users.

III. Policy Proposal: The Youth Digital Wellbeing Protection Act creates four layers of protection

Phase 1: Before Teens Can Use the App

• Clear warning: “This is NOT therapy or a replacement for professional help.”

• Parental consent verified for anyone under 18.

• Plain-language explanation of how the app uses and stores data.

Phase 2: App Safety Standards

• Mental health claims reviewed by licensed clinicians.

• Safety rating system (like movie ratings) for easy understanding.

• Regular, independent safety audits.

• Bias & Equity: test accuracy across racial, ethnic, LGBTQ+, neurodiverse, and linguistic subgroups; report subgroup performance gaps.

Phase 3: Protecting Teen Data

• No training AI on teen conversations without explicit permission.

• Teens can delete their data at any time.

Phase 4: When Things Go Wrong

• Automatic alerts to licensed counselors for crisis situations.

• Direct connection to human help when needed.

• School staƯ trained to use digital mental health tools responsibly.

Enforcement Mechanisms

• Independent Oversight Board: Cross-sector body including clinicians, technologists, youth advocates, and privacy experts

• Penalty Structure: Graduated fines from $10,000 to $50 million based on user base size and violation severity

• Whistleblower Protections: Safe harbors for employees reporting violations

IV. Addressing Key Concerns

Innovation vs. Protection

Strict rules don’t have to slow AI progress. Senators Alex Padilla and Peter Welch warn that unregulated AI companion apps create “unearned trust,” leading teens to share sensitive information, including self-harm thoughts, with chatbots untrained to respond (Senator Welch, 2025). Early research on Therabot, an AI mental health product, shows that keeping users safe requires clinical oversight and careful design.

Won’t This Kill Innovation?

No. Clear rules give ethical companies a competitive edge. When we built our chatbot, we implemented safeguards voluntarily; this Act would make those protections the standard, rewarding responsible innovation.

What About Equity?

AI mental health tools must serve all teens. The Act requires:

• Conduct bias and equity audits to test accuracy across racial, ethnic, LGBTQ+, neurodiverse, and language groups

• Accessibility plans must include multilingual access, aƯordability, and availability to ensure no community is excluded.

Is This Realistic?

Yes. COPPA updates in 2025 already require separate parental consent before using children’s data to train AI. This Act builds on existing laws and international models, reducing cost and speeding adoption.

Long-Term Vision & Personal Experience

The goal isn’t to remove AI from mental healthcare, but to ensure it helps without exploitation. The Nature Mental Health review warns that unregulated AI companions risk fostering dependency without safeguards. When building our depression-detection chatbot, we implemented escalation protocols, avoided storing identifiable data, and consulted school counselors not because the law required it, but because ethics did. The proposed Act would make such protections standard.

Conclusion

This is the moment to decide whether AI mental health tools will democratize support or exploit youth mental health needs. The Youth Digital Wellbeing Protection Act oƯers a path to innovation anchored in transparency, privacy, and human oversight. Teens deserve tools built to help, not harvest their pain.

Works Cited

Common Sense Media (2024). Teens and AI: Understanding the rise of AI companions. Common Sense Media.

Akin Gump (2025). New COPPA obligations for AI technologies collecting data from children. AI Law and Regulation Tracker.

American Psychological Association (2025). Using generic AI chatbots for mental health support: A dangerous trend. APA Services.

American School Counselor Association (2024). Student-to-school counselor ratios 2023–2024. ASCA.

Annie E. Casey Foundation (2024). Youth mental health statistics in 2024. AECF.

Boston Globe (2025). ChatGPT saved her life. Psychologists say it could also be dangerous. Published 17 July.

Centers for Disease Control and Prevention (2024). CDC data show improvements in youth mental health but need for safer and more supportive schools. CDC Newsroom.

European Union (2024). Artificial Intelligence Act (Regulation (EU) 2024/1689). OƯicial Journal of the European Union.

Legal Nodes (2023). How to process children’s data in AI apps in a compliant way. Published 2 November.

European Data Protection Board (2025). AI: Italian Supervisory Authority fines company behind chatbot Replika.

Mozilla Foundation (2023). Shady mental health apps inch toward privacy and security improvements, but many still siphon personal data. Published 2 May.

Nature Mental Health (2024). Risks and ethical challenges of AI companions in mental health support.

OECD.AI (2025). The therapeutic caveat: Prohibitions on manipulation and persuasion in the EU AI Act.

Senator Welch (2025). Senators demand information from AI companion apps following kids’ safety concerns, lawsuits.

Stanford HAI (2024). AI & society report: Regulatory sandboxes for responsible innovation.