Designing LLM Agent Systems
2 min read
Designing LLM Agent Systems
An agent is an LLM that can take actions — call tools, query APIs, run code — in a loop until it accomplishes a goal, rather than answering in a single shot.
The agent loop
Goal ─▶ ┌───────────────────────────────┐
│ 1. LLM reasons about state │
│ 2. Chooses a tool + arguments │◀── observation
│ 3. Tool executes │
│ 4. Result fed back as context │───┘
└──── repeat until done ─────────┘ ─▶ Final answer
This is ReAct (reason + act): the model decides what to do next each iteration.
Core components
- Tools — well-described functions (search, DB query, run code). Clear names + schemas are critical.
- Memory — short-term (the conversation) and long-term (a vector store).
- Orchestrator — the loop that runs the model, executes tools, handles errors and limits.
Single vs multi-agent
| Pattern | When |
|---|---|
| Single agent + tools | Most tasks — simpler, cheaper, easier to debug |
| Multi-agent | Distinct skills or parallel work; adds coordination overhead |
Prefer the simplest design that works. Multi-agent systems multiply cost, latency and failure modes.
The hard parts of production agents
- Reliability — models err; add validation, retries and guardrails around every tool call.
- Cost & latency — each loop is an LLM call; cap iterations and cache.
- Runaway loops — set a max-steps budget; detect repeated identical actions.
- Security — sandbox code execution, whitelist actions, never trust tool output blindly (prompt injection).
- Observability — log every reasoning step and tool call; you can't debug an agent you can't trace.
Takeaway: an agent is a control loop around an LLM. The intelligence is in the model; the engineering is in the tools, guardrails and observability.