The Rise of AI in Personal Health Conversations
In recent years, artificial intelligence (AI) tools like ChatGPT have become a go-to resource for many Americans seeking help with personal health matters. From discussing symptoms and medications to sharing lab results and mental health struggles, these platforms are being used more frequently than ever before. For many individuals, AI has become the first point of contact before consulting a physician—offering a faster, more accessible, and seemingly more private alternative to traditional healthcare interactions.
A January report from OpenAI revealed that among its 800 million active ChatGPT users, one in four enters a query about healthcare weekly. Over 40 million people per day ask ChatGPT questions related to health. This trend highlights how deeply AI is becoming integrated into everyday health decisions.
However, this growing reliance on AI raises important concerns about privacy and data protection. Many Americans assume that their health-related conversations are protected under the Health Insurance Portability and Accountability Act (HIPAA). Unfortunately, this assumption is outdated. HIPAA applies to doctors, hospitals, and insurers but does not cover consumer AI platforms. When patients share medical information with ChatGPT, they are interacting with a technology platform governed by its own privacy policies, not by the legal protections that apply to healthcare professionals.
This discrepancy creates a significant gap between perceived privacy and actual legal protection. Patients often believe they are having confidential discussions, but in reality, their information may be stored, analyzed, or even shared without their full understanding or consent.
The Evolution of Health Data Sharing
What makes this moment different from the early days of internet search is the depth and detail of the information being shared. Users are no longer typing a few keywords into a search engine. Instead, they are providing rich, narrative health histories that include timelines, medications, family history, and emotional context. In effect, they are creating something that resembles a medical record—except it exists entirely outside the traditional healthcare system.
Health data is highly valuable because it can predict current conditions, future risks, costs, and longevity. As more of this data is generated in places where traditional safeguards do not apply, the need for updated policies becomes increasingly urgent.
At the same time, a parallel transformation is happening inside the exam room. Ambient AI scribes, such as those developed by Suki and Optum, are now listening to doctor-patient conversations, transcribing them, and converting them into structured clinical documentation. These tools are introduced as a solution to physician burnout, and they do reduce the burden of documentation. However, they also change the nature of the medical record. Instead of a physician summarizing a visit, entire conversations are captured, processed, and stored, often by third-party systems.
The Impact on Medical Documentation
This shift has implications beyond efficiency. When every word in the exam room becomes data, documentation begins to blur into optimization. Companies like Optum are integrating these systems into broader analytics and revenue cycle workflows, linking what is said during a clinical encounter to coding, billing, and payer decision-making. What was once a private conversation between doctor and patient is increasingly part of a larger financial and actuarial pipeline.
The question is no longer just how care is delivered, but how it is recorded, interpreted, and ultimately monetized at the expense of physicians and consumers. This transformation underscores the growing complexity of the healthcare data ecosystem.
A New Health Data Supply Chain
Taken together, these trends reveal the emergence of a new health data supply chain—one that spans both outside and inside the clinical setting. Patients are voluntarily sharing highly sensitive information with AI platforms on one end, while ambient systems are passively capturing clinical conversations in unprecedented detail on the other. In both cases, patients often do not fully understand what is being collected, where it is stored, or who ultimately has access.
What begins as a conversation is rapidly becoming a permanent, queryable record. This is not a failure of technology; it is a failure of policy. HIPAA was designed for a healthcare system built around hospitals and insurers, not conversational AI and always-on ambient computing. It does not account for systems that simulate medical dialogue or continuously capture clinical interactions. Nor does it address the growing convergence of clinical data, consumer data, and financial analytics.
Even when consent is obtained, it is often buried in lengthy disclosures that few patients read or fully comprehend. Americans are not careless with their health information—they are operating under assumptions that no longer hold true. They believe they are having private, protected conversations when, in reality, they are participating in a rapidly expanding data ecosystem that operates largely outside the traditional boundaries of medical confidentiality.
The Path Forward
As artificial intelligence becomes more embedded in both everyday life and clinical care, the gap between expectation and reality will only widen. The solution is not to retreat from innovation but to catch policy up with practice. That means rethinking how health data is defined, who is responsible for protecting it, and how consent is obtained in an era where conversations themselves have become data.
Until then, Americans will continue to speak freely—inside and outside the exam room—without fully understanding who else may be listening.
Sreedhar Potarazu, MD, MBA, is an author, board-certified ophthalmologist, and clinical director at American Eye Care.
