AI Agent Best Practices
Agentic AI Design Patterns over Frameworks
In this article we are diving deep into AI Agents. If you have been feeling overwhelmed by the sheer number of frameworks, libraries, and tools promising to build “autonomous generic super-agents,” take a deep breath. ☁️
We have looked at dozens of teams building LLM agents across industries, and the secret to success isn’t complexity—it’s simplicity.
Workflows vs. Agents
First, let’s draw a line in the sand. At Anthropic and across the industry, we distinguish between two types of systems:
1. Workflows: Systems where LLMs and tools are orchestrated through predefined code paths. You know exactly what step comes next.
2. Agents: Systems where the LLM dynamically directs its own process. It decides what tool to use and when to stop.
Best Practice: Always try to build a Workflow first. Only move to an Agent when the problem is so open-ended that you cannot define the steps in code.
The Architecture Patterns
Let’s walk through the sketch note architectures (reference the diagram above).
1. The Building Block: The Augmented LLM
Before any fancy architecture, you have the unit. This is an LLM composed with retrieval (RAG), tools (APIs), and memory.
Tip: Focus on your “Agent-Computer Interface” (ACI). Your tool definitions should be as polished as your prompts.
2. Prompt Chaining
This is the “Hello World” of compound systems. You decompose a task into sequential steps.
The Pattern: Output of Step A -> Input of Step B.
Real World Example: You want to generate a blog post.
Call 1: Generate an outline.
Call 2: Critique the outline.
Call 3: Write the post based on the critique.
3. Routing
Not all inputs require the strongest (and most expensive) model. Routing classifies input to direct it to the right process.
The Pattern: Input -> Router (Classifier) -> Specialized Path.
Real World Example: Customer Support.
“I want a refund” -> Route to a rigid, policy-following workflow.
“My product is making a weird noise” -> Route to a technical troubleshooting agent.
4. Parallelization
LLMs are slow. Parallelization buys you speed and quality.
Sectioning: Breaking a large task (like checking a 100-page doc) into 10 chunks and processing them simultaneously.
Voting: Running the same prompt 3 times and having a 4th call pick the best answer to reduce hallucinations.
5. Orchestrator-Workers
This is where things get powerful for complex tasks.
The Pattern: A central “Orchestrator” LLM breaks down a plan and delegates sub-tasks to “Worker” LLMs. The Orchestrator synthesizes the results.
Real World Example: Coding Assistants. The Orchestrator looks at a Jira ticket and decides which 5 files need to be edited. It spins up 5 Worker instances to edit those files in parallel.
6. Evaluator-Optimizer
This creates a feedback loop that mimics human refinement.
The Pattern: Generator LLM -> Response -> Evaluator LLM (provides feedback) -> Generator LLM (revises).
Real World Example: Translation. An LLM translates a poem. An Evaluator checks for lost nuance. The Generator translates again, incorporating the notes.
When to go Autonomous?
Autonomous Agents operate in a loop:
Thought -> Action -> Observation.
They are powerful but hard to control. Use them for open-ended research or tasks where the “how” is impossible to predict.
Summary
Success isn’t about the most sophisticated system; it’s about the right system. Start with simple prompts, move to Chaining, then Orchestrators. Only build fully autonomous agents when necessary.
Keep building! ☁️👩💻
References: