HIPAA Compliance Guide

HIPAA-Compliant LLM Usage: Protect Patient Data When Using AI

Healthcare organizations are adopting AI tools at record speed, but sending Protected Health Information (PHI) to third-party AI APIs creates significant HIPAA risk. AI Privacy Gateway masks PHI before it leaves your infrastructure, enabling safe and compliant LLM usage.

TL;DR: Every time a healthcare provider pastes patient data into an AI tool, PHI leaves their control. AI Privacy Gateway automatically detects and masks PHI (names, medical record numbers, phone numbers, addresses, etc.) before it reaches the AI API. Deploy as a local proxy in 30 seconds.

HIPAA and AI APIs: The Compliance Challenge

Under HIPAA, covered entities and business associates must ensure that Protected Health Information (PHI) is safeguarded when shared with third parties. Using AI APIs presents several challenges:

✗ No Business Associate Agreement (BAA)

Many AI API providers do not offer BAAs, or only offer them for enterprise plans. Without a BAA, sending PHI to an AI API violates HIPAA.

✗ Data De-identification Required

HIPAA's Safe Harbor method requires removing 18 specific identifiers from PHI. Manual de-identification is error-prone and time-consuming.

✗ No Audit Trail

Standard AI API usage lacks the access logs, disclosure tracking, and audit controls required by HIPAA.

✗ Data Retention Risk

AI providers may retain prompts for model improvement. HIPAA requires defined data retention and destruction policies.

The 18 HIPAA Identifiers: How the Gateway Handles Them

The gateway automatically detects and masks the 18 HIPAA-designated identifiers before they reach the AI API:

Names Patient and family member names
Phone Telephone numbers
Email Email addresses
Address Street address, city, county, ZIP
SSN Social Security Numbers
MRN Medical record numbers
Health Plan Health plan beneficiary numbers
Account Account numbers
License License plate numbers
Device Device identifiers and serial numbers
URL Web URLs
IP IP addresses
Biometric Biometric identifiers (custom rule)
Photos Full-face photos (custom rule)
Dates Dates of birth, admission, discharge
Age Ages over 89 (custom rule)
Note: The gateway covers 14 identifiers out-of-the-box. The remaining 4 identifiers (biometric data, photos, etc.) can be protected with custom regex rules. The gateway's custom rules system lets you add any PHI pattern specific to your healthcare organization.

How AI Privacy Gateway Enables HIPAA-Compliant LLM Use

Clinician App
→
Privacy Gateway
(PHI masker)
→
AI API
(de-identified data)
✓ PHI removed before transmission ✓ No BAA required for masked data ✓ Full audit trail of masking events

How De-identification Works

The gateway implements a HIPAA-compliant de-identification workflow:

  1. Detection: Regex + NER dual engine scans every request for the 18 HIPAA identifiers
  2. Masking: Each identifier is replaced with an anonymous placeholder (e.g., [PATIENT_NAME_1], [PHONE_1])
  3. Forwarding: Only de-identified data reaches the AI API — no PHI transmitted
  4. Restoration (optional): If needed, the gateway can restore placeholders in the AI response with original values, but only on your local machine

Setup Guide for Healthcare Organizations

Docker (HIPAA-compliant local deployment)
docker run -d \
  --name ai-privacy-gw \
  -p 9999:9999 \
  -v ./vault_data:/app/vault_data \
  -e TARGET_LLM=https://api.openai.com \
  ghcr.io/gunxueqiu6/ai-privacy-gateway:lite

For healthcare deployments, we recommend running the gateway on-premise or in a HIPAA-compliant cloud environment. Configure your AI API clients to use http://localhost:9999/v1 as the base URL.

Adding Custom PHI Patterns

Healthcare organizations often have facility-specific PHI patterns (internal MRN formats, facility codes, etc.). The gateway's admin dashboard lets you add custom regex rules without restarting:

Custom Rule Examples
# Medical Record Number pattern
Pattern: MRN-\d7
Type: Custom PHI
Placeholder: [MRN_1]

# Internal Facility Code
Pattern: FAC-\w4-\d3
Type: Custom PHI
Placeholder: [FACILITY_1]

Frequently Asked Questions

Does masking PHI remove the need for a BAA with the AI provider?

If PHI is properly de-identified according to HIPAA's Safe Harbor method (removing all 18 identifiers), the resulting data is no longer considered PHI and a BAA may not be required. However, you should consult with your compliance team to confirm your specific use case.

Is the gateway itself HIPAA-compliant?

The gateway is a tool to help you achieve HIPAA compliance, not a certified solution. It provides the technical safeguards required by HIPAA — access controls, encryption at rest, audit logging, and automatic PHI de-identification. You should still conduct a risk assessment and implement appropriate administrative safeguards.

Can I deploy this in a HIPAA-compliant cloud environment?

Yes. The gateway runs on any infrastructure. Deploy it in your HIPAA-compliant AWS, Azure, or GCP environment. The gateway itself is stateless and can be deployed behind a VPN or private network.

Does it work with medical AI tools like scribes or CDSS?

Yes. Any tool that makes API calls to an LLM can be routed through the gateway. This includes AI scribes, clinical decision support systems, medical coding tools, and patient communication platforms.

How is the vault data encrypted?

The SQLite vault is encrypted using AES-256-GCM. The encryption key is auto-generated on first start and stored locally. You can also set your own key via the VAULT_ENCRYPT_KEY environment variable.

Can I audit what PHI was masked?

Yes. The admin dashboard provides real-time masking statistics and a searchable log of detected entities. For HIPAA audit purposes, you can export masking logs from the dashboard.

Deploy HIPAA-Compliant LLM Access

Mask PHI automatically. Keep patient data private. Deploy in 30 seconds.

Get Started Free Try the Demo