An AI agent uses a model, tools and feedback to work toward a goal. Learn how agents work, when to use them, and how to evaluate reliability and cost.
Start with the core questions, then examine the examples and tradeoffs below.
An AI agent is a system that uses a model to choose actions toward a goal, interacts with tools or an environment, and uses the resulting feedback to decide what to do next. Its behavior also depends on application code, permissions, state, and stopping rules. The model alone is not the complete agent.
A chatbot describes a conversational interface and may or may not take actions. A fixed workflow follows steps specified in code. An agent can choose some steps or tool calls dynamically. These categories overlap: a chat interface can contain an agent, and a workflow can delegate a bounded task to one.
An agent's cost includes every model call, repeated context, tool use, retries, and supporting infrastructure involved in a task. Failed runs and human correction also matter. Compare cost per accepted task, not just a model's per-token price or the cost of its first response.
Make the task boundary explicit before making the system autonomous.
A task starts with a goal and a definition of completion. The application supplies relevant context and available tools. The model proposes a response or tool call; application code checks whether the call is valid and permitted, executes it when allowed, and returns the result. The next inference step can use that feedback.
The loop ends when the task succeeds, a budget or time limit is reached, a tool cannot proceed, or a person must decide. Store explicit task state rather than relying on conversational prose as the sole record of what happened. When a run is resumed, distinguish a completed action from an action that was only proposed.
Anthropic's architecture guidance distinguishes predefined workflows from systems where the model directs its next steps. That distinction is useful when deciding how much flexibility a task needs. A predictable sequence can often remain ordinary code with model calls at selected points.
Sources: Anthropic: Building effective agents
Tools and state turn a generated suggestion into an operational system.
Treat the tool contract as part of the product. Describe accepted inputs, possible side effects, errors, and the evidence returned on completion. Application code should enforce access checks and validate arguments. Model-generated text should not be able to grant itself permissions.
Memory can mean short-lived task context, persisted application state, or retrieved records. Those are different mechanisms with different retention and access requirements. Specify what is stored, who can retrieve it, and how stale or incorrect information is corrected.
| Component | Responsibility | A useful check |
|---|---|---|
| Model | Propose the next response or action | Can it select and populate the tool correctly? |
| Tools | Perform bounded operations | Are inputs validated and side effects documented? |
| State | Record progress and results | Can a resumed run avoid repeating completed actions? |
| Policy | Enforce permissions and limits | Can an unapproved action be blocked outside the model? |
| Evaluation | Measure task outcomes | Can a test detect a plausible but incorrect success claim? |
Start with a task whose output and failure cases can be inspected.
A document assistant can gather approved files, extract fields, and draft a comparison with source references. Completion can require that every extracted value maps back to a document passage. A support assistant can gather account context and draft a resolution, while refunds or account changes remain subject to explicit policy.
A coding agent can propose a patch and run project checks. Passing tests is useful evidence but does not prove that the change is secure or that it matches the intended product behavior. A desktop agent can operate applications, but it also needs to handle focus changes, dialogs, partial progress, and actions that cannot safely be repeated.
Use ordinary software for stable calculations and deterministic rules. An agent is most useful where the next step depends on information discovered during execution; it need not control every part of a process.
The bill follows the execution trace, not the user's single message.
Track a task identifier across model calls and tools. Attribute input, output, cached usage, tool charges, and retries to that task. Keep elapsed time, accepted completion, and escalation outcomes alongside the costs. This makes a cheaper but less reliable configuration visible.
Illustrative example: a task makes eight model calls averaging $0.003 each and uses tools costing $0.006 in total. Its direct cost is $0.030. If ten runs consume $0.30 and only eight produce accepted results, the direct cost per accepted result is $0.0375, before operator time and other infrastructure. These are made-up rates used to explain the calculation.
Repeatedly sending an entire conversation can make later steps more expensive. Keep the facts a step needs, preserve evidence required for verification, and avoid discarding state needed to identify completed actions. Test context reduction against task outcomes rather than assuming shorter context is always safe.
A confident final answer is not a completion signal.
Define the desired final state and unacceptable side effects. For a file task, check the artifact. For a database task, check the intended records and unintended changes. For a research task, inspect source support. Include failed tools, missing permissions, ambiguous inputs, and recovery after interruption.
Anthropic's agent-evaluation guidance describes combining code-based checks, model judgments, and human review. Use deterministic checks where a result can be checked exactly, and calibrate judgment-based grading against examples reviewed by people. Repeated runs help reveal behavior that a single successful demonstration misses.
Log the evidence needed to diagnose a failure without indiscriminately retaining confidential data. A task can succeed with an inefficient route, or fail after many individually valid steps. Review both the trace and the final outcome before changing the model or prompt.
A planner, extraction step, and final verifier can have different requirements.
A constrained extraction step may be eligible for a different model from a difficult planning step. Define quality and feature requirements for each role, but also rerun the complete task: changing one step can alter the distribution of inputs seen by the next step.
o10 connects agent inference to routing through its OpenAI-compatible endpoint. Its Squad mode uses a planner, workers, and judge for multi-step work; its DeepShell product is a desktop agent in beta. Read each product's current integration and availability details before choosing a deployment path.
A multi-agent design is not automatically better than a single agent. Coordination adds calls and failure modes. Add specialized roles when evaluations show that they improve accepted outcomes enough to justify the extra cost and latency.
Write the input contract, completion condition, allowed tools, and operations that require human approval.
Compare the agent with the simpler workflow or manual process it is meant to improve.
Evaluate completed outcomes, incorrect actions, tool failures, interruption, and repeated runs.
Bound steps, elapsed time and spend. Provide a stop control and a clear route to human review.
Track task success, cost, latency, escalations and incidents together. Re-evaluate after changes to models or tools.
Architecture guidance with original illustrative cost arithmetic. Examples describe possible system designs, not reported customer outcomes. Product descriptions follow o10's capability specification; availability and integration details remain on the product pages.
Squad: multi-agent inference · DeepShell: desktop AI agent · Agent workload economics
No. An agent can use one model. Multiple models or agents are architectural choices that should be justified by better measured outcomes, specialized requirements, or cost advantages after accounting for coordination overhead.
The level of supervision depends on the task and permissions. Some bounded operations can be automated; consequential or ambiguous actions may require review. Enforce those limits in the application rather than relying only on a prompt.
Not by itself. A retrieval pipeline can supply information to a fixed model call. An agent may choose when to retrieve, what to search for, and what action to take based on the results.
Set per-task step, time and spend limits; monitor retries and repeated context; and compare cost per accepted outcome. Model routing can help when alternatives pass the relevant evaluations, but it does not replace task-level budgets.
Squad is o10's multi-agent inference mode for multi-step tasks. DeepShell is its desktop agent product in beta. Both are related to agent use, while the routing endpoint provides inference access for applications that manage their own agent loop.
Paste a week of traffic. Get the number that books the audit.
See what you're overpaying →