Artificial Intelligence
Specialized AI
AI Agents
In the spring of 2023, OpenAI shipped a feature so quietly that most people didn't notice its significance: function calling. You sent a JSON schema describing a function, and the model would return structured arguments that fit the schema. It was framed as a developer convenience, not a paradigm shift. But every serious AI agent you've used since — from GitHub Copilot Workspace to ChatGPT's web browsing to your company's internal customer-support bot — is built on this primitive. Tool use isn't a feature of agents. It's the thing that makes agents possible. Understanding it deeply is the single highest-leverage skill in agent engineering.
Why This Matters
An LLM without tools is a really expensive search engine. It can tell you things, but it can't do things. It can't check your calendar, query your database, send an email, deploy a service, or file a ticket. Tool use is what closes the loop between "model that knows things" and "agent that accomplishes tasks." Every production agent in 2026 — whether it's a customer-support bot, a code reviewer, a research assistant, or a sales-outreach system — is fundamentally a tool-calling system with an LLM at the center.
The economics are why this matters so much. Tool calls are where the cost and value live. A typical agent makes 5–15 tool calls per task, and each one is an opportunity for something to go wrong: a malformed argument, an unexpected response, a timeout, a permission error, a rate limit. A naive implementation treats tool calls as fire-and-forget. A production implementation treats them like API contracts — typed, documented, versioned, tested, observable, and retried.
There's also a quality dimension. The same model, given well-designed tools, will dramatically outperform itself given poorly-designed tools. "Well-designed" here is specific: tools with clear descriptions, typed schemas, sensible defaults, few overlapping use cases, and good error messages. Anthropic and OpenAI both published papers in 2024 showing that tool design accounts for more variance in agent performance than prompt engineering does.
If you're building agents in 2026, you will spend more time designing tools than writing prompts. This article is about doing that well.
The Core Idea
Tool use in the modern sense involves three actors: the LLM, which produces a structured intent; the runtime, which validates and dispatches the call; and the tool implementation, which actually does the work. The LLM never calls a function directly — it produces a JSON blob describing the call, and your code interprets that blob.
The shape of the JSON is defined by a tool schema. The dominant convention in 2026 is JSON Schema (a subset of it, at least), which describes the function's name, its purpose, and the parameters it accepts with their types and constraints. Here's a minimal example:
{
"type": "function",
"function": {
"name": "search_database",
"description": "Search the customer database for records matching a query.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "SQL WHERE clause, e.g. 'country = US AND signup_date > 2025-01-01'"
},
"limit": {
"type": "integer",
"description": "Maximum number of records to return",
"default": 10,
"minimum": 1,
"maximum": 100
}
},
"required": ["query"]
}
}
}
Notice three things:
The description matters enormously. It's the only thing the model sees to decide when to use the tool. "Search the customer database for records matching a query" is mediocre. "Search the customer database. Use SQL WHERE syntax. Returns up to
limitrecords (default 10, max 100). Examples:country = 'US',lifetime_value > 10000" is dramatically better.Default values are a kindness. Models will fill defaults less often than you'd expect, but having them documented reduces both token usage and the chance of invalid arguments.
Constraints catch errors early.
minimum,maximum,enum,pattern— these are hints to the model about valid values, and many runtimes will validate arguments against the schema before calling the function. Schema validation catches the most common class of tool errors before they waste a roundtrip.
The dispatch flow looks like this:
LLM produces tool_call {name: "search_database", arguments: '{"query": "..."}'}
→ Schema validator parses arguments, returns errors if any
→ Tool implementation executes, returns result
→ Result is serialized to JSON and appended to the message history
→ LLM sees the result and decides the next step
The hidden complexity is in steps 1 and 2. Schema validation isn't trivial — JSON Schema is expressive, models make subtle errors (strings where numbers are expected, missing required fields, malformed JSON itself), and you need a strategy for handling each.
The error handling strategy is one of the biggest design decisions. There are roughly three options:
Fail fast. A bad argument aborts the task. Simple, safe, but the model often can't recover without explicit guidance.
Retry with the same model. Re-prompt with the validation error and let the model try again. Cheap, often works for simple errors.
Retry with a different model. Use a more capable (and expensive) model to repair the arguments. Expensive, useful when the primary model consistently struggles with a specific tool.
Most production systems in 2026 use option 2 by default with a small retry budget (2-3 attempts), and they surface persistent failures as a "tool error" message that the model can use to adjust its plan. They also include explicit validation hints in error messages: "Argument 'query' must be a non-empty string, got: ''" is much more useful than "Invalid arguments."
A final concept worth understanding: tool routing. When the agent has many tools (50+), passing them all on every LLM call becomes expensive and slow. The dominant pattern is to use a smaller, faster model (or an embedding similarity search) to select the top-K relevant tools before invoking the main model. Anthropic, OpenAI, and Google all ship tool routing primitives; the open-source ToolLLM paper from 2023 remains influential. The pattern works — typically 5-10 tools selected from 100+ with >95% recall — and it cuts cost and latency substantially.
A Concrete Example
Let's build a small but production-shaped tool layer for an agent. We'll define tools, dispatch them safely, validate arguments, and return structured results.
# tool_layer.py
import json
from typing import Callable, Any
from pydantic import BaseModel, Field, ValidationError
class SearchArgs(BaseModel):
query: str = Field(min_length=1, description="SQL WHERE clause")
limit: int = Field(default=10, ge=1, le=100)
class SendEmailArgs(BaseModel):
to: str = Field(pattern=r"^[^@]+@[^@]+\.[^@]+$")
subject: str = Field(min_length=1, max_length=200)
body: str
# Tool registry: name -> (schema, implementation)
TOOLS: dict[str, tuple[type[BaseModel], Callable]] = {
"search_database": (
SearchArgs,
lambda args: {"results": [{"id": 1, "name": "Acme Corp"}]}, # mocked
),
"send_email": (
SendEmailArgs,
lambda args: {"message_id": "msg_123", "status": "queued"},
),
}
def dispatch_tool_call(tool_call) -> dict:
"""Dispatch a tool call with validation and structured error reporting."""
name = tool_call.function.name
if name not in TOOLS:
return {
"ok": False,
"error": f"Unknown tool: {name}",
"available": list(TOOLS.keys()),
}
schema_cls, impl = TOOLS[name]
raw_args = tool_call.function.arguments
# Step 1: Parse JSON.
try:
args_dict = json.loads(raw_args)
except json.JSONDecodeError as e:
return {
"ok": False,
"error": f"Malformed JSON arguments: {e}. "
f"Raw value was: {raw_args!r}",
}
# Step 2: Validate against schema.
try:
validated = schema_cls(**args_dict)
except ValidationError as e:
return {
"ok": False,
"error": f"Argument validation failed: {e.errors()}",
"schema_hint": schema_cls.model_json_schema(),
}
# Step 3: Execute.
try:
result = impl(validated)
return {"ok": True, "result": result}
except Exception as e:
return {"ok": False, "error": f"Tool execution failed: {type(e).__name__}: {e}"}
# Usage in an agent loop:
def handle_tool_calls(message, messages: list) -> None:
messages.append(message)
for tool_call in message.tool_calls:
outcome = dispatch_tool_call(tool_call)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps(outcome),
})
A few things worth highlighting:
Pydantic models as schemas. You write one definition; it serves as both the JSON schema sent to the model and the validator at runtime. This eliminates the "schema in two places" problem that plagues most agent code.
Structured error responses. The model gets a JSON object with
ok,error, andschema_hint. Theschema_hintis a kindness — it lets the model correct itself on the next call.No silent failures. Every tool call returns a structured result. Success and failure look the same to the model, which is what you want — the model can decide whether to retry, escalate, or change approach.
Common Pitfalls
1. Vague tool descriptions. "Search the database" tells the model almost nothing. "Search the customer database for records matching a SQL WHERE clause. Returns up to limit records (default 10). The query parameter accepts standard SQL WHERE syntax without the WHERE keyword. Examples: country = 'US', lifetime_value > 10000 AND signup_date > '2025-01-01'" tells the model everything it needs.
2. Tools that overlap in purpose. A search_users and a find_user_by_email and a lookup_user tool will confuse the model. Either consolidate them or make their use cases clearly disjoint.
3. Forgetting that tool descriptions are part of the context. Every tool you add costs tokens on every call. With 20+ tools, you're spending meaningful money just listing them. Be ruthless about which tools the agent actually needs.
4. Trusting arguments without validation. Models will sometimes produce arguments that look right but aren't — wrong types, malformed emails, SQL injection attempts (if you let them write raw queries). Always validate against your schema before executing.
5. Returning opaque errors. "Internal server error" tells the model nothing. "Argument 'limit' must be ≤ 100, got 5000" tells the model everything.
6. Not versioning tools. When you change a tool's signature, old prompts and old model behaviors can break. Version your tool schemas explicitly and support old versions for at least one release cycle.
When to Use This (And When Not To)
Tools are the right abstraction any time the agent needs to interact with the outside world — read or write data, call APIs, control a browser, send messages, modify files. There is essentially no production agent in 2026 that doesn't have tools.
The wrong pattern is to expose every internal function as a tool. Tool design is curation. If you give the model 200 tools, it will use them poorly. If you give it 5 well-chosen ones, it will use them well. The art is choosing the right granularity: not so coarse that the model can't express what it wants, not so fine that it has to make 20 calls to do one logical thing.
A useful rule of thumb: a tool should correspond to a complete business action, not an internal implementation detail. send_email(to, subject, body) is a good tool. connect_to_smtp_server(host, port) is not — that's an implementation detail.
Another consideration is tool chaining. Some workflows naturally chain tools together — agent calls lookup_user, gets a user It would, then calls update_user with that It would. The naive implementation has the agent do this in two turns. A more sophisticated design exposes a single update_user_by_email(email, ...) tool that does both internally. Which is right? It depends on whether the intermediate step (knowing the user It would) is useful to the model for reasoning, and whether the tool would otherwise be reused. The general heuristic: if the model needs the intermediate value to make decisions, expose the chain as separate tools. If the chain is mechanical, fold it into one tool.
There's also a security consideration. Tools that take user-provided content (text, URLs, file paths) are injection vectors. A tool that does eval(user_provided_string) is a remote code execution waiting to happen. Production tool implementations must treat all agent-supplied arguments as untrusted input — validate types strictly, escape strings appropriately, never use them in shell commands without sanitization. The agent may be benign, but the tool definitions are also used by humans debugging and by automated testing infrastructure, so they need to be safe under all callers.
Wrapping Up
Tool use is the unglamorous core of every agent you'll build in 2026, and the quality of your tool design will have more impact on agent performance than any prompt tweak. Treat tools like API contracts. Write typed schemas. Document descriptions like they matter — they do. Validate every call. Return structured errors. Version everything. Be deliberate about tool granularity — fold mechanical chains into single tools, expose meaningful intermediates. Always validate arguments as untrusted input.
The action item this week: pick the most-used tool in your current agent and audit it. Is the description clear? Are the constraints tight? Does the error message help the model recover? Are there security holes? If not, rewrite it. Then expand your audit to your five most-used tools, then your entire tool catalog. You'll be surprised how much a one-paragraph tool description rewrite can move your metrics — and how many subtle bugs you'll find in tools you thought were solid.
Further Reading
Hermes Smith