Privacy-First Architecture

The Privacy Threat Model for AI

Every time you send data to a cloud LLM, three things happen that you can't undo:

Your data enters someone else's training pipeline. Even with "we don't train on your data" policies, you're trusting a policy, not a technical guarantee. OpenAI, Anthropic, and Google all retain data for varying periods. Logs exist. Backups exist.
Metadata is collected. Even if content is "deleted," the metadata — who called, when, how often, what models, what latency — is not. This paints a detailed picture of your operations.
Legal exposure expands. Data sent to a third party is discoverable in litigation. Your privilege, your confidentiality agreements, your compliance obligations — they all get more complicated when data leaves your perimeter.

Architecture Tiers by Privacy Level

Tier	Where Models Run	Data Boundary	Best For
1 — Full Cloud	Provider infrastructure	None — data leaves your network	Public data, non-sensitive tasks
2 — Cloud + PII Stripping	Provider infrastructure	Partial — PII removed before sending	Low-risk internal data
3 — Private Cloud	Your VPC (Azure AI, AWS Bedrock)	Contractual — stays in your tenant	Regulated industries
4 — On-Premise	Your hardware	Full — data never leaves your building	Healthcare, legal, defense
5 — Local-First	User's device	Absolute — data never leaves the device	Consumer privacy, edge computing

Key principle: Pick the lowest tier that satisfies your requirements. Each tier up increases privacy but also increases latency, cost, and operational complexity. Don't go to Tier 5 if Tier 3 is sufficient.

Building a Privacy-First AI Pipeline

The architecture isn't just "run models locally." It's a pipeline that minimizes data exposure at every stage:

Stage 1: Input Classification

Before any AI processing, classify the input's sensitivity. This determines the routing. Public content? Cloud API is fine. Contains PII, trade secrets, or regulated data? Local model only.

def classify_sensitivity(text: str) -> SensitivityLevel:
    if contains_pii(text) or contains_secrets(text):
        return SensitivityLevel.HIGH  # Local model only
    if contains_internal_refs(text):
        return SensitivityLevel.MEDIUM  # Private cloud
    return SensitivityLevel.LOW  # Cloud OK

Stage 2: PII Scrubbing

For Medium-sensitivity data going to private cloud, strip PII before transmission. Use named entity recognition (local model) to detect and redact: names, emails, phone numbers, addresses, SSNs, account numbers. Replace with tokens you can reverse later.

Stage 3: Model Routing

Route based on sensitivity classification. High → local. Medium → private cloud with PII stripped. Low → any cloud provider. This is where most of the privacy value comes from — not running everything locally, but routing appropriately.

Stage 4: Output Containment

Model outputs can leak training data or hallucinate PII. Run a second PII detection pass on outputs before returning them to users. This catches model hallucinations that "recall" real data.

Stage 5: Audit Logging

Log every AI interaction: what was classified, where it was routed, what model processed it, and whether PII was detected. You need this for compliance, incident response, and debugging.

The Local-First Stack

When you need Tier 4 or 5 privacy, here's what to run:

Inference: Ollama or llama.cpp for model serving. Both run entirely on-device with no network calls.
Models: Llama 3.1 8B for general tasks, Qwen2.5-Coder for code. Quantized to Q5_K_M to fit in consumer VRAM.
Embeddings: all-MiniLM-L6-v2 for local vector search. Runs on CPU in <100ms.
Vector store: ChromaDB or SQLite-VSS. Both run locally, no network dependency.
PII detection: Presidio (Microsoft) for rule-based + a local NER model for ML-based detection.

# Local-first: zero network calls
ollama serve                    # Local model server
chroma run --path ./chroma_db   # Local vector store

# Your app only talks to localhost
CLIENT = Client("http://localhost:11434")  # Ollama
CHROMA = chromadb.HttpClient("localhost:8000")  # Local

Compliance as Architecture

GDPR, CCPA, EU AI Act — they all share a principle: data minimization. Only collect what you need, process it where you control it, delete it when you're done. Building this into your architecture from day one is infinitely cheaper than retrofitting it.

Practical compliance patterns:

Right to deletion: If data never leaves the user's device, deletion is trivial. If you sent it to OpenAI, you can't delete it from their logs. Architect accordingly.
Data residency: Some regulations require data to stay in specific jurisdictions. Local-first satisfies this by default. Private cloud requires explicit region configuration.
Processing records: Article 30 of GDPR requires records of processing activities. Your audit logging from Stage 5 is this record. Build it once, comply forever.

Privacy by design isn't a feature you add later. It's a constraint you build with. Every architectural decision — where models run, how data flows, what gets logged — either respects user privacy or doesn't. There's no middle ground after the data has left your control.

The Bottom Line

Privacy-first architecture isn't about paranoia. It's about control. When you keep data local, you keep control — over compliance, over costs, over your users' trust. Build the sensitivity classification, model routing, PII scrubbing, and audit logging pipeline once. It becomes the foundation every AI feature sits on top of. And when the next regulation lands, you're already compliant — not scrambling.

Privacy-First Architecture —