AI Agents — How They Work

An agent is just an LLM running in a loop. You give it a goal and access to tools, and it repeatedly decides what to do next until the goal is met or it gives up. That's the entire concept. Everything else — frameworks, orchestrators, "agentic architectures" — is plumbing around this loop.

The Core Loop: Observe, Reason, Act

The pattern that makes agents work is called ReAct (Reasoning + Acting). Each iteration looks like this:

1. Observe — the agent reads the current state: the original goal, conversation history, and results from any previous tool calls
2. Reason — the model decides what to do next, often producing a brief chain-of-thought
3. Act — the model emits a structured tool call (or a final answer)
4. Observe result — the tool output is appended to context, and the loop continues

The loop terminates when the model produces a final response instead of a tool call, or when you hit a guardrail (max iterations, timeout, cost cap). The model is choosing, at each step, whether to keep going or stop.

Tool/Function Calling

Tools are how agents interact with the world. You describe available tools in a schema — name, description, parameters with types. The model reads these descriptions and, when it decides to use one, emits a structured JSON call specifying which tool and what arguments.

{
  "name": "search_database",
  "arguments": {
    "query": "customer cancellation rate Q2",
    "limit": 10
  }
}

Critically — the model never executes anything. It outputs text that happens to be a structured tool call. Your code parses that call, validates it, executes it, and feeds the result back. That boundary between "model suggests" and "code executes" is where all the security lives. The model is making requests; your application is the one with actual permissions.

Planning and Decomposition

More capable agents break complex goals into subgoals. Ask an agent to "prepare a quarterly report" and a good one will decompose that into: query the database, aggregate metrics, generate charts, draft narrative, compile into a document.

Planning happens in two flavors. Upfront planning generates a full plan before acting — clean but brittle when early steps fail. Incremental planning decides one step at a time based on accumulated context — more adaptive but can wander. The best agents do both: sketch a rough plan, then adjust as they go. Replanning on failure is what separates useful agents from fragile scripts.

Memory: Short-Term and Long-Term

Short-term memory is the context window. Everything the agent has seen in this session — the goal, tool calls, results, reasoning — lives here. When it fills up, the agent loses information. This is a hard constraint, not a tunable parameter.

Long-term memory means persisting information across sessions, typically in a vector store or structured database. An agent might save facts about user preferences, project context, or lessons learned. On the next run, it retrieves relevant memories and injects them into the prompt. This works but adds complexity and latency, and the retrieval can surface irrelevant or outdated information.

Multi-Agent Patterns

The idea is appealing: specialized agents collaborating, one for research, one for coding, one for review. In practice, multi-agent setups introduce coordination overhead, context duplication, and failure modes that are harder to debug than a single agent with multiple tools.

Multi-agent patterns make sense when you genuinely need different model configurations (different system prompts, different tool sets, different temperature settings) or when you need to enforce separation of concerns for security reasons. For most applications, a single agent with well-designed tools is simpler and more reliable.

Failure Modes

Agents fail in characteristic ways you need to design for:

• Infinite loops — the agent keeps calling the same tool with the same arguments, or oscillates between two states without making progress
• Cascading errors — one bad tool call produces a confusing result, the agent misinterprets it, makes a worse call, and spirals
• Context exhaustion — long-running agents fill their context window with tool results and lose track of the original goal
• Hallucinated tool calls — the model invents tool names or parameters that don't exist
• Goal drift — the agent subtly redefines the goal based on intermediate results and solves the wrong problem

Guardrails: Containing the Loop

The core risk with agents is excessive agency — giving a model the ability to take actions with real-world consequences without adequate constraints. Every guardrail exists to address this:

• Max iterations — hard cap on loop cycles. If the agent hasn't finished in 20 steps, something is wrong
• Cost caps — set a maximum token/dollar budget per run. An unbounded agent with GPT-4 access can burn hundreds of dollars in minutes
• Tool allowlists — the agent can only call tools you explicitly provide. No dynamic tool discovery
• Human-in-the-loop — require human approval before irreversible actions (sending emails, deleting data, deploying code)
• Sandboxing — run tool execution in isolated environments. If the agent generates and runs code, that code runs in a container, not on your production server
• Input/output validation — validate tool call arguments before execution and sanitize results before feeding them back

The principle is simple: the more powerful the tools, the tighter the guardrails. A read-only agent needs fewer constraints than one that can modify databases and send communications.

Key Takeaways

• An agent is an LLM in a loop: observe the state, reason about the next step, call a tool, observe the result, repeat
• The model never executes tools — your code does. That boundary is where security is enforced
• Planning (decomposition + replanning on failure) separates useful agents from demos
• Short-term memory is the context window; long-term memory requires external storage and retrieval
• Multi-agent patterns add complexity that is rarely justified — start with a single agent and well-designed tools
• Guardrails (max iterations, cost caps, tool allowlists, human-in-the-loop, sandboxing) are not optional when agents have real-world tool access

Next, we'll look at vector databases — the storage layer that gives agents and RAG systems the ability to search over large knowledge bases by meaning rather than keywords.