Tool Design
Tool Curation
Managing tool registries, role-based subsets, sprawl prevention, and dynamic tool selection
Last verified 2026-09-24
Summary
Curate task-specific tool subsets from a centralized registry rather than loading everything at once. An with 100 endpoints shouldn't show all 100 tools; expose a read-only set for auditors, a mutation set for admins, a focused set per workflow. This improves agent reasoning, reduces cost, and enforces permission boundaries. Left unmanaged, a tool registry accumulates duplicate and overlapping tools - sprawl - that degrades selection accuracy the same way an oversized single request does.
- Registry structure: metadata (category, auth, ), handler, schema
- Subsets: read-only, mutations, admin, dangerous (role-based)
- Guidance, not a hard ceiling: OpenAI calls "fewer than 20 functions available at the start of a turn" a soft suggestion; Anthropic's tool search tool notes selection accuracy degrades past 30-50 available tools
- Dynamic selection: keyword/BM25 filtering, embedding-based ranking, or Anthropic's server-side Tool Search Tool once a registry is large
- Namespacing:
billing__get_invoiceavoids name collisions across domains - Ownership: every tool has an owner, a status (active/deprecated/experimental), and a removal path
An API might expose 100 endpoints. An agent should not see all 100 tools at once. Tool curation means curating a task-specific subset: read-only tools for auditors, mutation tools only for admins, a focused set for a specific workflow.
Agent frameworks route tool calls by matching the user's request against each tool's name and description, so every extra tool raises the chance of routing to the wrong one. There's no single hard ceiling across vendors: OpenAI describes "fewer than 20 functions available at the start of a turn" as a soft suggestion, and Anthropic notes that Claude's tool-selection accuracy degrades once a session exceeds roughly 30-50 available tools. Treat 20 as a review threshold regardless of provider - above it, curate a subset, namespace, or introduce dynamic selection rather than loading everything.
Tool Registries
A tool registry is a centralized collection of all available tools with metadata. The agent system queries the registry to build task-specific subsets.
Registry Structure
// tools/registry.ts
import { z } from "zod";
interface ToolMetadata {
name: string;
description: string;
schema: z.ZodSchema;
handler: (params: any) => Promise<any>;
category: "read" | "write" | "admin" | "dangerous";
requiresAuth: boolean;
rateLimit?: { requestsPerMinute: number };
owner: string;
status: "active" | "deprecated" | "experimental";
}
const toolRegistry: Record<string, ToolMetadata> = {
search_issues: {
name: "search_issues",
description: "Search for issues by text, assignee, or status...",
schema: searchIssuesSchema,
handler: searchIssuesHandler,
category: "read",
requiresAuth: false,
owner: "platform-team",
status: "active",
},
delete_issue: {
name: "delete_issue",
description: "Permanently delete an issue (cannot be undone)...",
schema: deleteIssueSchema,
handler: deleteIssueHandler,
category: "dangerous",
requiresAuth: true,
rateLimit: { requestsPerMinute: 1 },
owner: "platform-team",
status: "active",
},
};
export const getAllTools = () => Object.values(toolRegistry);
export function getToolsForUser(userId: string): ToolMetadata[] {
const user = await getUserPermissions(userId);
return Object.values(toolRegistry).filter((tool) => {
if (tool.status !== "active") return false;
if (tool.requiresAuth && !user.authenticated) return false;
if (tool.category === "admin" && !user.isAdmin) return false;
if (tool.category === "dangerous" && !user.isDangerousActionsAllowed) return false;
return true;
});
}Namespacing
Prefix tool names by domain to prevent collision and make ownership unambiguous - double underscore is the conventional separator:
[
{
"name": "billing__get_invoice",
"description": "Get invoice details from the billing system..."
},
{
"name": "billing__create_invoice",
"description": "Create a new invoice in the billing system..."
},
{ "name": "docs__get_document", "description": "Get a document from the document store..." }
]The namespace prefix makes the owning system clear even when the tool name itself is generic (get_invoice vs. billing__get_invoice).
Role-Based Tool Kits
Different roles need different tools. Create curated kits per role, defined at a shared specification level rather than per agent instance:
const toolKits = {
auditor: {
description: "Read-only access for compliance audits",
tools: ["search_issues", "get_issue", "list_users", "get_user", "search_audit_logs"],
},
developer: {
description: "Create and update issues, code review workflows",
tools: [
"search_issues",
"create_issue",
"update_issue",
"search_pull_requests",
"create_pull_request",
],
},
admin: {
description: "Full access including dangerous operations",
tools: [...Object.keys(toolRegistry)],
},
};
export function getToolKitForRole(role: string): ToolMetadata[] {
const kit = toolKits[role];
if (!kit) throw new Error(`Unknown role: ${role}`);
return kit.tools.map((name) => toolRegistry[name]).filter((t) => t !== undefined);
}Why Sprawl Happens
Left unmanaged, a tool registry grows monotonically: a new capability needs a tool, the existing set isn't checked for overlap, a slightly different variant gets added instead of extending the original, experimental tools are never removed, and deprecated endpoints stay registered because removing them feels risky. The result is a namespace with several tools that do roughly the same thing under different names - the agent may call the wrong one, call all of them, or fail to call any confidently.
Dynamic Tool Selection
When a registry has more tools than a single session should see, don't load every definition upfront. Three approaches, in increasing order of setup cost:
Keyword Filtering
The simplest approach for explicit categories - no infrastructure required:
function selectToolsByKeywords(userQuery: string, allTools: ToolMetadata[]): ToolMetadata[] {
const keywords = userQuery.toLowerCase().split(/\s+/);
return allTools.filter((tool) => {
const toolText = `${tool.name} ${tool.description}`.toLowerCase();
return keywords.some((kw) => toolText.includes(kw));
});
}Active Tools (Vercel AI SDK)
const result = await generateText({
model,
tools: allTools,
activeTools: ["search_issues", "create_issue"], // Only these are loaded
prompt: userMessage,
});Anthropic's Tool Search Tool
For Claude, the server-side tool search tool discovers and loads tool definitions on demand instead of front-loading everything. Mark tools you don't need up front with defer_loading: true, include a tool_search_tool_bm25_20251119 (natural-language queries) or tool_search_tool_regex_20251119 (Python-regex patterns) entry in tools, and Claude searches names, descriptions, and argument fields to discover and expand the tools it needs - typically loading only 3-5 tools per turn out of a catalog of thousands. No beta header is required; it's a standard tool type. See Anthropic's tool search tool docs for the request/response shape.
Embedding-Based Selection at Scale
Keyword and regex search match on literal text; embedding-based selection ranks tools by semantic similarity between the user's message and each tool's description, catching relevant tools even when the wording differs. This is the right choice when you're building the ranking yourself rather than delegating to a provider's server-side search:
- Embed every tool description once at startup; cache the vectors (file or Redis) so restarts don't recompute them.
- On each turn, compute cosine similarity between the user's message and the cached vectors; select the top ~12 candidates as a starting point, tuned against your tool set and .
- Keep a small "always-active" set (web search, a
search_toolsmeta-tool) that bypasses ranking, so the agent can always discover what the top-N selection missed. - Maintain a
relatedToolsmap for write operations that depend on a read tool the embedding wouldn't surface on its own - e.g.invoices_createneedscustomers_listpre-loaded even though "create an invoice for $500" has no semantic overlap with "list customers." Keep this map to observed dependencies; don't add transitive ones speculatively. - Expose a
search_toolsmeta-tool so the model can look outside the top-N selection when it needs a tool that didn't rank:
server.registerTool(
"search_tools",
{
description:
"Search for a tool by name or capability. Use when you can't find the tool you need.",
inputSchema: z.object({ query: z.string() }),
},
async (params) => {
const results = await toolIndex.search(params.query, { maxResults: 5 });
return { content: [{ type: "text", text: JSON.stringify(results) }] };
},
);Token savings are the main payoff: a 50-tool list at ~500 tokens shrinks to a 12-tool selection at ~120 tokens per request. The tradeoffs are embedding latency (tens of milliseconds per request), a cold-start cost on the first request after a restart, and coverage - a tool outside the top-N is unreachable except through search_tools.
Read-Only vs Mutating Kits
Distinguish safe (read) from risky (write) tools when assembling an agent's tool set:
export const readOnlyKit = Object.values(toolRegistry).filter((t) => t.category === "read");
export const mutatingKit = Object.values(toolRegistry).filter((t) =>
["read", "write"].includes(t.category),
);
export const dangerousKit = Object.values(toolRegistry); // requires approval: see Human Approval
const auditAgent = agent({ tools: readOnlyKit, model: "claude-sonnet-5" });
const adminAgent = agent({
tools: dangerousKit,
model: "claude-opus-5-5",
requireApproval: (toolName) => toolRegistry[toolName].category === "dangerous",
});See Human Approval for the approval contract behind requireApproval.
Tool Deprecation and Evolution
Tag tools with stability metadata and filter production/beta selections by it:
export function selectToolsForProduction(allTools: ToolMetadata[]): ToolMetadata[] {
return allTools.filter((t) => t.status === "active");
}In descriptions, note deprecated tools and their replacement: "DEPRECATED: Use \search_issues` instead. Removed 2026-12-01."` Never keep a deprecated tool registered out of fear of breakage - removal is safer than an agent discovering and calling a stale tool.
Rate Limiting at the Registry Level
async function executeToolWithRateLimit(toolName: string, params: any): Promise<any> {
const tool = toolRegistry[toolName];
if (!tool) throw new Error(`Unknown tool: ${toolName}`);
if (tool.rateLimit) {
const count = toolCallCounts.get(toolName) || 0;
if (count >= tool.rateLimit.requestsPerMinute) {
return {
isError: true,
content: [
{ type: "text", text: `Rate limit exceeded: ${tool.rateLimit.requestsPerMinute}/min.` },
],
};
}
toolCallCounts.set(toolName, count + 1);
}
return await tool.handler(params);
}Sprawl Prevention Checklist
Before adding a new tool:
- Search the registry for an existing tool that covers this capability
- If a similar tool exists, is the new one genuinely different enough to warrant a separate entry?
- Does the new tool have an owner assigned?
- Is the tool name unique and does it follow the namespace convention?
- Will this tool be added to an approved role kit, or only used experimentally?
- Is the target agent's tool set already at or near the ~20-tool review threshold?
Review tool call metrics monthly - a tool with zero calls in 90 days is a removal candidate; a tool with a high error rate needs a description fix.
Project Layout
src/
├── tools/
│ ├── registry.ts # All tool metadata
│ ├── kits.ts # Role-based subsets
│ ├── selection.ts # Dynamic selection logic
│ └── handlers/
└── test/
├── registry.test.ts
└── selection.test.tsSee also
- Idempotency and Safety - the read/write/destructive classification that role-based kits filter on
- Human Approval - gating the dangerous kit behind approval
/docs/tool-design/cross-framework-portability- adapting registry tools to frameworks- (Anthropic tool design guide)
- (Anthropic tool search tool)
- (OpenAI function calling guide)