Building Secure AI Products

This is the synthesis lesson. We've covered how LLMs work, how to call their APIs, how to run them in production, and where they're vulnerable. Now we put it together: an actual reference architecture for building an LLM application that's secure by design, not by hope.

Design Principles

Before diving into layers, internalize these principles. They should inform every architectural decision:

Assume the model is compromised. Design your system so that a prompt injection — successful manipulation of the model's behavior — cannot lead to a security incident on its own. The model is a component you don't fully control. Treat it that way.

Least agency. Give the model the minimum tools, permissions, and data access it needs. Every additional capability is attack surface. If your chatbot doesn't need to delete records, don't provide a delete tool.

No implicit trust in model output. The model's response is an untrusted input to the rest of your system. Validate it, sanitize it, and enforce schemas before acting on it.

Defense in depth. No single layer should be the only thing preventing a security incident. Each layer should work independently so that one failure doesn't cascade.

Fail closed. When something unexpected happens — malformed model output, a tool call that doesn't match the schema, an authorization check you can't perform — deny the action rather than allowing it.

The Layered Architecture

Input Layer

Everything starts here. The user's request enters your system and must be validated before it reaches the model.

Input validation — enforce length limits, reject obviously malformed inputs, and strip control characters. This isn't primarily about injection defense (injections look like valid text); it's about basic hygiene and preventing DoS.

PII detection — scan inputs for PII and redact before sending to a third-party model API if your compliance requirements demand it. Use regex for structured PII (SSNs, credit cards) and model-based detection for unstructured PII.

Rate and cost limits per user — enforce before you incur API costs. Per-user limits on requests per minute, tokens per day, and total spend per billing period. This is your denial-of-wallet defense.

def validate_input(user_id: str, text: str) -> str:
    if len(text) > MAX_INPUT_LENGTH:
        raise InputTooLong()
    if not rate_limiter.allow(user_id):
        raise RateLimitExceeded()
    text = pii_detector.redact(text)
    return text

Context Layer

In a RAG application, this is where you retrieve documents and build the context that accompanies the user's query.

Per-tenant retrieval filtering — every query against a shared vector database must include a tenant filter applied before the model sees results. Not after. A hard filter on the query, enforced in your code.

Authorization on retrieval — apply the same access control rules you'd enforce on a direct document access request. If a user can't view a document through your UI, they shouldn't see it through RAG either.

Provenance tracking — tag each retrieved chunk with its source document and access control metadata for auditing.

Model Layer

This is the model interaction itself — system prompt construction and the API call.

System prompt hygiene — keep it concise. Don't include internal documentation, API keys, or database schemas. Assume the system prompt can and will be extracted.

No secrets in prompts — never pass API keys, connection strings, or auth tokens in prompts or tool descriptions. Credentials live in server-side tool execution code only.

Instruction hierarchy — use system vs user roles to establish priority. A partial defense, not a guarantee, but it raises the bar.

Output Layer

The model has responded. Before that response reaches a user or downstream system, it must be validated.

Schema validation — when you expect structured output (JSON, tool calls), validate against a strict schema. Reject responses that don't conform. This catches both hallucinated structures and injection attempts that try to produce malformed output.

Output encoding — if model output is rendered in HTML, apply the same output encoding you'd use for any user-generated content. Escape HTML entities. Use Content-Security-Policy headers. The model can produce <script> tags whether or not you asked it to.

Content filtering — scan for harmful content, hallucinated URLs, or data that looks like it came from another tenant's context. This is especially important in multi-tenant systems where context bleed could expose one customer's data to another.

Action Layer

This is the most security-critical layer. When the model decides to call a tool, your code is about to take an action in the real world.

Tool allowlists — the model can only call tools you've explicitly registered. No dynamic tool discovery, no model-generated tool definitions. The tool set is static and reviewed.

Per-action authorization independent of the model — this is the key architectural insight. When the model calls delete_record(id=42), your tool execution code must verify that the current user has permission to delete record 42. This check happens in your code, using your existing authorization system. The model's judgment is irrelevant to this decision.

def execute_tool(tool_name: str, args: dict, user: User):
    if tool_name not in ALLOWED_TOOLS:
        raise ToolNotAllowed(tool_name)
    
    # Authorization at the tool boundary
    tool = ALLOWED_TOOLS[tool_name]
    if not authz.check(user, tool.required_permission, args):
        raise Unauthorized()
    
    # Validate arguments against schema
    validated = tool.schema.validate(args)
    
    # Execute in sandbox if applicable
    return tool.execute(validated)

Sandboxing — tools that execute code, run queries, or interact with external systems should run in isolated environments. Containers, VMs, or serverless functions with minimal permissions and no access to your broader infrastructure.

Human approval — for irreversible or high-impact actions (sending emails to customers, modifying financial records, deploying code, deleting data), require explicit human approval before execution. Show the user exactly what the model wants to do and let them confirm or reject.

Observability Layer

You can't secure what you can't see.

Log every request: input, prompt (PII-redacted), retrieved documents, model response, tool calls and results, token counts, latency, cost, and authorization decisions. Set up alerts for anomalous patterns: cost spikes, unusual tool calls, repeated authorization failures, injection-like inputs. Retain logs for incident reconstruction.

Testing: Red-Team Your Own App

Don't wait for attackers to find your vulnerabilities.

Injection test suites — maintain a collection of prompt injection payloads and run them in CI. Test both direct and indirect injections. Verify that architectural defenses hold even when the model complies with the injection.

Red-teaming — periodically have team members attempt to break your application through creative prompt manipulation. Document findings and update defenses.

Authorization boundary testing — verify that tool-level authorization denies actions for unauthorized users regardless of what the model requests.

Key Takeaways

• Assume the model is compromised — design your architecture so a successful prompt injection alone cannot cause a security incident
• Authorization must live in your code at the tool boundary, never in the prompt — the model's judgment is irrelevant to authorization decisions
• Layer defenses independently: input validation, per-tenant retrieval filtering, system prompt hygiene, output schema validation, tool allowlists, per-action authorization, and sandboxing
• Never put secrets in prompts or tool descriptions — credentials belong in server-side tool execution code
• Fail closed: when model output is unexpected or authorization can't be verified, deny the action
• Test your defenses in CI with injection test suites and regular red-teaming — don't wait for attackers

This wraps up the AI track. You now have the mental model to build LLM applications that are useful, affordable, and defensible — not just demos that work when everyone behaves.