Guide

LLM data privacy explained

A support agent pastes a customer’s full medical claim into ChatGPT to draft a reply. The model provider logs the prompt. A week later, another user’s retrieval query surfaces a paraphrase of that diagnosis. That is not a hypothetical edge case — it is what happens when LLM data privacy is treated as an afterthought. Large language models amplify ordinary SaaS mistakes: text you send becomes training-adjacent telemetry, long-lived embeddings, and searchable logs. This guide maps privacy risks across training, fine-tuning, RAG ingestion, inference, and observability; explains regulatory hooks (GDPR, CCPA, sector rules); covers data minimization, redaction, access control, and retention; walks through a Harbor Support ticket assistant privacy architecture; compares deployment patterns in a decision table; lists common pitfalls; and ends with a production checklist. For safety rails around harmful outputs, see our LLM guardrails primer; for retrieval design, see RAG fundamentals and vector databases.

What “LLM data privacy” actually covers

Privacy in LLM systems spans four distinct surfaces. Confusing them leads to signing a “no training on customer data” clause while still logging every prompt to S3 for five years.

Training and fine-tuning data — corpora, SFT pairs, RLHF transcripts, and evaluation sets that may contain names, account numbers, or health details scraped from production.
Inference inputs and outputs — user prompts, uploaded files, tool results, and model replies transiting third-party APIs or your own GPUs.
Retrieval corpora — documents chunked, embedded, and indexed in vector stores; often the richest PII concentration in enterprise RAG.
Operational telemetry — traces, prompt/completion logs, feedback thumbs, and annotation queues used for debugging and quality improvement.

Privacy vs security vs safety

Security stops unauthorized access (authn/z, encryption). Safety stops harmful generations (jailbreaks, toxic content). Privacy limits who may see whose data and for how long, even when access is authorized. A perfectly secured system can still violate privacy by retaining prompts indefinitely or retrieving another customer’s invoice into the wrong session.

Where sensitive data leaks in production

Prompt and context stuffing

Engineers paste entire JSON records into prompts “for context.” Each field — email, phone, internal user ID — copies to the model provider and your log pipeline. Data minimization means sending only fields the model needs: ticket category and error code, not the full CRM export.

RAG cross-tenant retrieval

Vector indexes without tenant isolation can return chunks from Company A when Company B asks a similar question. Fixes: per-tenant namespaces, metadata filters on every query, and row-level security in the source database that propagates to chunk tags.

Memorization and regurgitation

Models fine-tuned on production chats may reproduce training snippets verbatim under the right prefix. Mitigate with deduplication, PII scrubbing before SFT, differential privacy (rare in practice), and post-training extraction evals that hunt for secrets.

Logs, traces, and vendor subprocessors

LangSmith, Datadog LLM Observability, and hosted inference dashboards store full prompts by default. Each vendor becomes a subprocessor under GDPR Article 28 — you need DPAs, data residency choices, and retention caps.

Human review queues

RLHF and “flagged conversation” workflows expose raw user text to annotators. Role-based access, audit trails, and geographic restrictions on reviewer pools are mandatory for regulated data.

Regulatory and contractual hooks (practical, not legal advice)

Laws do not mention “embeddings,” but they do mention personal data processing. Map your LLM flows to these obligations and involve counsel for your jurisdiction.

GDPR (EU/UK) — lawful basis for processing, data subject access/deletion requests, DPIAs for high-risk profiling, cross-border transfer mechanisms (SCCs), and 72-hour breach notification.
CCPA/CPRA (California) — disclosure of categories sold/shared, opt-out of sale, sensitive personal information limits, and service-provider contracts that prohibit secondary use.
HIPAA, GLBA, PCI — sector rules may forbid sending certain fields to public APIs at all; air-gapped or BAA-covered hosting becomes a hard requirement.
Enterprise DPAs — customers increasingly demand zero-retention inference, EU-only regions, and explicit prohibitions on using their prompts for model improvement.

Document a Record of Processing Activities for each LLM feature: what categories of personal data flow where, retention periods, and deletion procedures including vector index tombstoning.

Data minimization and redaction pipelines

Treat every byte sent to a model as a liability. Build a preprocessing layer that runs before prompts, RAG ingestion, and log export.

Detection and masking

Combine regex (credit cards, SSN patterns), NER models, and allowlists for internal IDs. Replace detected spans with stable tokens ([EMAIL_1]) so the model can reason about structure without seeing values. Store a reversible vault mapping only on your side if you must rehydrate for human agents — never send the vault to the LLM vendor.

Field-level policy

Declare which CRM columns are never-LLM, masked-LLM, or full-LLM. Enforce in code, not documentation. A single engineer bypassing the filter for debugging becomes your breach story.

Output filtering

Scan completions for PII before returning to users or writing to logs. Models hallucinate phone numbers; block patterns that match detected input spans or high-entropy digit runs.

RAG and vector database privacy controls

Retrieval systems duplicate source documents into embeddings. Privacy design starts at ingestion:

Source authorization — only index documents the retrieval user could read in the origin system; sync ACL changes on delete/update events.
Chunk metadata — attach tenant_id, classification, and source_doc_id to every vector; filter on all three at query time.
Encryption — encrypt vectors and payloads at rest; use customer-managed keys (CMEK) when contracts require it.
Deletion — implement hard deletes in the index when users exercise erasure rights; soft deletes are not enough for regulators.
Re-embedding after redaction — if you scrub PII from source docs, re-chunk and re-embed; stale vectors retain old secrets.

For hybrid search architectures, apply the same filters to keyword and vector legs. BM25 can leak via exact token match even when semantic search is filtered.

Inference logging, retention, and user rights

Default posture: log the minimum needed to debug, not everything.

Structured traces — store latency, model ID, token counts, and hashed session IDs; omit prompt text unless sampled under explicit consent.
TTL policies — 7–30 days for debug logs, 0 days for prompts in high-sensitivity modes; automate S3 lifecycle rules and observability vendor settings.
Access-on-access — engineers view redacted traces by default; full-text access requires break-glass approval with audit log.
Deletion workflows — tie user account deletion to prompt log purge, vector tombstones, and fine-tuning dataset exclusion lists.

Publish a clear privacy notice explaining whether humans review chats, which providers process data, and how to request deletion.

Worked example: Harbor Support ticket assistant

Harbor Support routes billing and shipping tickets through an LLM that drafts replies and suggests macros. Privacy architecture:

Ingress gateway — strips payment card numbers and government IDs via Microsoft Presidio-style detectors before any model call.
Tenant-scoped RAG — policy PDFs and macro libraries embed with merchant_id metadata; retrieval queries always include filter: merchant_id == session.merchant.
Model routing — EU merchants hit Azure OpenAI in West Europe with zero-retention deployment; U.S. merchants use a self-hosted Llama endpoint on Harbor VPC GPUs.
Logging — production traces store token counts and intent labels only; 2% sample of redacted prompts goes to a 14-day QA bucket with engineer SSO and MFA.
Human escalation — when agents edit drafts, diffs feed a weekly fine-tuning set that passes through the same PII scrubber; no raw ticket text enters the SFT pool.
DSAR automation — merchant admin portal triggers deletion across Postgres sessions, S3 trace samples, and Pinecone namespaces within 30 days.

Result: support quality gains without copying full PANs into OpenAI logs or cross-leaking one merchant’s refund policy into another’s session.

Deployment decision table

Pattern	Privacy profile	Trade-offs
Public API (default logging)	Lowest; vendor may retain for abuse monitoring	Fastest to ship; unacceptable for HIPAA/strict DPAs
Enterprise API (zero retention, region lock)	Medium-high; contractual guarantees, still off-prem	Higher cost; verify subprocessors and audit reports
Self-hosted open weights	High; data stays in your VPC	GPU ops burden; you own patching and safety stacks
On-device / edge SLM	Highest for user inputs; no network exfil	Limited capability; model updates harder to ship
Federated / local-only RAG	Strong for corpus privacy; queries stay local	Sync complexity; weaker models on laptop hardware

Common pitfalls

“We don’t train on your data” checkbox theater — inference logging and RAG indexes still store personal data; read the full DPA.
Debug mode in production — verbose prompt logging left on after an incident; rotate keys and purge logs immediately.
Shared vector index — one Pinecone index for all customers with only prompt-level instructions to “ignore other clients.”
Annotator access sprawl — contractors worldwide reading unredacted chats containing health or financial data.
Shadow IT Copilot — employees paste customer data into consumer chatbots with no corporate controls; solve with approved tools and DLP.
Forgetting embeddings on deletion — user deleted from app DB but vectors remain searchable for months.
Over-broad tool permissions — agent can call CRM export APIs and stuff entire tables into context; scope tools to single-record fetches.

Production checklist

Map personal data flows for training, RAG, inference, and logs in a processing register.
Sign DPAs with every model and observability vendor; document subprocessor list.
Implement ingress PII detection and field-level never-LLM policies.
Enforce tenant isolation in vector indexes with metadata filters on every query.
Choose region-appropriate hosting and zero-retention modes per data class.
Set automated log TTLs; default traces to redacted structured events.
Build DSAR deletion across app DB, logs, vectors, and annotation sets.
Run periodic extraction evals after fine-tuning to detect memorized secrets.
Restrict human review pools; audit every full-prompt access.
Train engineers: no production prompts in personal ChatGPT accounts.
Publish user-facing privacy disclosures for AI features.
Tabletop a breach scenario: which logs and indexes must you purge in 72 hours?

Key takeaways

LLM privacy is four problems — training data, inference, RAG corpora, and telemetry — not one vendor checkbox.
Minimize before you tokenize; redact at ingress and scan at egress.
RAG without tenant filters is a cross-customer leak waiting to happen.
Retention policy matters as much as encryption; logs are the silent breach surface.
Match deployment pattern to regulatory class: public API for marketing copy, VPC or edge for regulated PII.