Artificial intelligence is transforming healthcare operations at a pace that existing compliance frameworks were never designed to accommodate. HIPAA — written in 1996 and last substantially updated in 2013 — does not mention AI, machine learning, or large language models. Yet every AI system that touches patient data must comply with it. Understanding where HIPAA applies, where it does not, and what leading healthcare organizations are doing to govern AI responsibly is now a board-level competency.
This guide is written for Chief Compliance Officers, Chief Information Security Officers, and healthcare executives who are deploying or evaluating AI systems. It covers the regulatory framework, the specific risks AI introduces, and the practical governance steps that distinguish compliant deployments from liability exposure.
Legal Disclaimer
This guide is for informational purposes only and does not constitute legal advice. Consult qualified healthcare legal counsel before making compliance determinations for your organization.
HIPAA and AI: The Regulatory Framework
HIPAA establishes three rules relevant to AI deployments: the Privacy Rule, the Security Rule, and the Breach Notification Rule. Each applies differently depending on how AI is used and what data it processes.
The Privacy Rule and AI
The Privacy Rule governs how Protected Health Information (PHI) can be used and disclosed. PHI is defined broadly — it includes any individually identifiable health information, whether in oral, written, or electronic form, that relates to a patient's past, present, or future health condition, healthcare provision, or payment for healthcare.
For AI systems, the Privacy Rule triggers in three critical scenarios:
- Training data: If an AI model is trained on patient data — even de-identified data — the de-identification process must meet the HIPAA Safe Harbor (18-element removal) or Expert Determination standard. Pseudonymization alone is insufficient.
- Inference: When a live AI model processes a patient's data to generate predictions, recommendations, or documentation, it is "using" PHI. This use must be for a permitted purpose under the Privacy Rule — typically Treatment, Payment, or Operations (TPO).
- Model outputs: AI-generated outputs that include or derive from PHI (a clinical note, a coding suggestion, a risk score for a specific patient) are themselves PHI and must be handled accordingly.
The Security Rule and AI Infrastructure
The Security Rule establishes administrative, physical, and technical safeguards for electronic PHI (ePHI). AI systems introduce several novel security considerations that the Rule does not directly address but that covered entities must manage:
- Model inversion attacks: Sophisticated adversaries can sometimes reconstruct training data from a model's weights. If a model was trained on ePHI, the model weights themselves may be considered ePHI, requiring the same access controls and encryption as any other ePHI asset.
- Prompt injection: In LLM-based clinical systems, malicious inputs can cause the model to disclose information from its context window — including other patients' data if multiple records are loaded simultaneously. AI-specific security testing is required.
- API security: AI systems typically expose APIs. Each API endpoint that accepts or returns ePHI must be assessed under the Security Rule's technical safeguard requirements: authentication, transmission encryption, access logging, and audit controls.
Business Associate Agreements: The First Line of Defense
Any AI vendor that creates, receives, maintains, or transmits ePHI on behalf of a covered entity is a Business Associate (BA) and must execute a Business Associate Agreement (BAA) before any data sharing occurs. This is not optional — it is an absolute legal requirement under 45 CFR §164.308(b).
For AI vendors specifically, your BAA must address several provisions that standard BAA templates often omit:
Critical AI-Specific BAA Clauses
| Clause | Why It Matters | What to Require |
|---|---|---|
| Training data use prohibition | Vendors may seek to use your PHI to improve their models — a secondary use requiring patient authorization | Explicit prohibition on using client PHI for model training without written authorization |
| Subprocessor disclosure | AI vendors routinely use cloud infrastructure, transcription services, and third-party APIs that become sub-BAs | Full list of subprocessors with processing locations and individual BAA representations |
| Data residency | PHI processed overseas may violate state law and creates additional jurisdictional risk | US-only processing requirement with written certification |
| Breach response timeline | HIPAA requires notification within 60 days; AI system breaches are complex and require early coordination | Vendor notification to covered entity within 24–48 hours of confirmed breach |
| Model audit rights | You must be able to demonstrate to OCR that AI outputs are appropriate and non-discriminatory | Right to audit model performance data, including disparity analysis by protected class |
Conducting an AI-Specific HIPAA Risk Assessment
The HIPAA Security Rule requires covered entities to conduct regular risk assessments of all systems that process ePHI. Standard risk assessment frameworks (NIST SP 800-30, OCR's SRA Tool) were not designed for AI systems and require significant adaptation. Here is a structured approach:
Step 1: AI System Inventory
Before you can assess risk, you need a complete inventory of every AI system in your environment that touches patient data. This is harder than it sounds — AI capabilities are embedded in EHR platforms, imaging systems, laboratory informatics, and productivity tools that clinicians may have adopted without IT involvement. Conduct a comprehensive discovery exercise, including shadow IT detection.
Step 2: Data Flow Mapping
For each AI system, map exactly what PHI flows in, where it is processed, where outputs are stored, and who has access. Pay particular attention to intermediary data stores — many AI systems create temporary embeddings, context stores, or training queues that contain PHI and are outside standard backup and access control frameworks.
Step 3: AI-Specific Threat Modeling
Add the following AI-specific threat vectors to your standard threat model:
- Prompt injection attacks (for LLM-based systems)
- Model inversion / membership inference attacks
- Data poisoning (for systems with online learning capabilities)
- Adversarial input attacks (particularly relevant for imaging AI)
- Vendor-side breaches that expose your ePHI in vendor infrastructure
Building an AI Governance Framework for Healthcare
HIPAA compliance is the floor, not the ceiling, of responsible AI governance in healthcare. Leading health systems are building comprehensive AI governance frameworks that address clinical safety, algorithmic fairness, transparency, and operational oversight — well beyond what HIPAA requires.
Establishing an AI Governance Committee
Every health system deploying clinical AI should have a cross-functional AI Governance Committee with representation from: Clinical Leadership, Legal/Compliance, IT Security, Privacy Officer, Clinical Informatics, Risk Management, and — critically — frontline clinicians who use the systems. This committee should meet at minimum quarterly and review new AI deployments before go-live.
Clinical Model Validation Requirements
AI models used in clinical decision support, documentation, or coding must undergo formal validation before deployment. At minimum, validation should demonstrate:
- Performance equivalence or superiority to the workflow it replaces
- Absence of clinically significant disparities across demographic subgroups (race, ethnicity, sex, age, payer)
- Defined confidence thresholds below which the model defers to human judgment
- Documented failure modes and clinician override mechanisms
How OnPoint Approaches HIPAA-Compliant AI
At OnPoint, compliance is architectural — not additive. Every component of the PMaaS™ platform is designed from the ground up for healthcare's most demanding regulatory requirements:
- SOC 2 Type II certified: Independent third-party audit of our security, availability, processing integrity, and confidentiality controls — renewed annually.
- HITRUST CSF validated: The gold standard for healthcare IT security frameworks, requiring assessment across 19 control domains.
- US-only data processing: All PHI is processed and stored on AWS GovCloud infrastructure in the continental United States. No overseas data routing.
- No training use of client data: Your patients' data is never used to train or improve our models without explicit written authorization.
- Full BAA provided: Our HIPAA-compliant BAA is provided at contract signing and covers all OnPoint subprocessors.
- 256-bit AES encryption: All data at rest and in transit is encrypted to AES-256 standards.
Conclusion: Compliance as Competitive Advantage
Healthcare organizations that build rigorous AI compliance frameworks are not just managing legal risk — they are building competitive advantage. Clinicians, patients, and payers increasingly scrutinize the AI systems in their healthcare interactions. Organizations that can demonstrate rigorous governance, transparent model behavior, and ironclad privacy protections will attract and retain patients and providers at a meaningful rate.
The regulatory environment will also tighten. The HHS Office for Civil Rights has signaled increasing enforcement attention on AI systems, and several states have enacted or are considering AI-specific healthcare legislation that goes beyond HIPAA. Organizations building strong governance frameworks today are positioned to absorb these requirements with minimal disruption.
Start with the fundamentals: complete your AI inventory, execute BAAs for every vendor, conduct an AI-specific risk assessment, and establish governance oversight. Then build upward from that foundation.