Designing LLM Agent Systems
2 min read
Designing LLM Agent Systems
An agent is an LLM that can take actions — call tools, query APIs, run code — in a loop until it accomplishes a goal, rather than answering in a single shot.
The agent loop
Goal ─▶ ┌───────────────────────────────┐
│ 1. LLM reasons about state │
│ 2. Chooses a tool + arguments │◀── observation
│ 3. Tool executes │
│ 4. Result fed back as context │───┘
└──── repeat until done ─────────┘ ─▶ Final answer
The model decides what to do next each iteration based on prior results. This is ReAct (reason + act).
Core components
- Tools — well-described functions the model can call (search, DB query, send email, run code). Clear names + schemas are critical; the model picks tools from their descriptions.
- Memory — short-term (the running conversation) and long-term (a vector store of past facts).
- Orchestrator — the loop that runs the model, executes tool calls, handles errors and enforces limits.
- Planning — decompose a complex goal into steps (sometimes a separate planning call).
Single vs multi-agent
| Pattern | When |
|---|---|
| Single agent + tools | Most tasks — simpler, cheaper, easier to debug |
| Multi-agent (specialised sub-agents) | Distinct skills or parallel work; adds coordination overhead |
Prefer the simplest design that works. Multi-agent systems are powerful but multiply cost, latency and failure modes.
The hard parts of production agents
- Reliability — models make mistakes; add validation, retries and guardrails around every tool call.
- Cost & latency — each loop is an LLM call; cap iterations and cache aggressively.
- Loops & runaway — set a max-steps budget and detect repeated identical actions.
- Security — a tool-using model is an attack surface. Sandbox code execution, whitelist actions, and never trust tool output blindly (prompt injection).
- Observability — log every reasoning step, tool call and result. You cannot debug an agent you cannot trace.
Design checklist
- Each tool has a precise description + input schema
- Max-iteration / token budget enforced
- Tool errors returned to the model gracefully (not crashes)
- Dangerous actions sandboxed or human-approved
- Full trace of steps logged for debugging
Takeaway: an agent is a control loop around an LLM. The intelligence is in the model; the engineering is in the tools, guardrails and observability around it.