Scoring Framework

Summary

Quantifies agent-readiness on a 0–30 scaled spectrum across 10 concrete dimensions: API, CLI, MCP, Discovery, Auth, Error Handling, Tool Design, Context Files, Testing, and Retrievability. Each dimension scores 0–3 (Not Implemented → Excellent), then the applicable total is scaled to 30. Provides a baseline score, identifies gaps, and tracks progress as you modernize. Scale: 0–7 (Human-only) to 23–30 (Agent-first).

• 0 = Not implemented (blocker)
• 1 = Basic (weak, human-oriented)
• 2 = Good (agent-ready, most use cases work)
• 3 = Excellent (production, resilient, cross-framework)
• N/A = Genuinely inapplicable (rare)

Agent-readiness is a spectrum. The scoring framework quantifies it across 10 concrete dimensions, giving you a measurement today, a clear view of gaps, and a way to track progress.

Why Score?

Ambiguity that humans resolve through intuition becomes failure modes at scale with agents. When an agent cannot ask for clarification, every implicit assumption, missing description, or hidden error path becomes a blocker. A 0–3 score per dimension tells you exactly where to invest: optimize the few surfaces agents actually touch, deprioritize human-only features.

Repository Audit vs Public-Site Scan

The Surface score is a source-backed review across code, documentation, runtime contracts, and tests. Public scanners such as Is Agentic, Agent Readiness, and Google Lighthouse's Agentic Browsing category observe the deployed website from the outside. They are complementary, but their scores are not interchangeable. See Public Scanners for how to read and record their output alongside a Surface audit.

Use a public scan to catch deployment reality: blocked crawlers, missing discovery files, unreadable HTML, or a capability that is not actually reachable. Use the Surface audit to find the source-level cause, assess private and authenticated surfaces, and decide whether a recommendation fits the product. Record the scanner, report URL, and scan timestamp as evidence; do not copy its headline number into the 0–30 Surface score.

The 0–3 Scale

Each dimension scores from 0 to 3:

• 0 (Not implemented) - Feature missing or unsuitable for agents. This dimension is a blocker.
• 1 (Basic) - Feature exists but weak. Human-oriented; terse; missing context agents need. Agents can use it with heavy prompting.
• 2 (Good) - Solid implementation. Agent-oriented descriptions, structured data, clear examples. Most agent use cases work.
• 3 (Excellent) - Production-ready for agents. Optimized for token efficiency, cross-framework, resilient error handling, advanced patterns (OAuth, MCP Tasks, delegation). Agents perform reliably with minimal prompt engineering.

The 10 Dimensions

1. API Surface - How well your HTTP API is described for agent tool generation. OpenAPI quality, operationIds, agent-oriented descriptions, Arazzo workflow support.

2. CLI Design - How well your CLI tool works with agent automation. JSON output, exit codes, schema introspection, input hardening (path traversal defense), SKILL.md packaging.

3. MCP Server - Whether and how well you expose an MCP server. Tool count, annotations, resources, RFC 9728 protected-resource metadata, pagination, testing with InMemoryTransport.

4. Discovery & AEO - How discoverable, readable, governable, and callable your project is by agents. llms.txt, AGENTS.md, JSON-LD schema, Markdown content negotiation, robots.txt, Content Signals, API Catalog, MCP Server Cards, Agent Skills indexes, OAuth metadata, Web Bot Auth, and optional commerce protocols.

5. Authentication - Whether clients can obtain and use credentials appropriate to the acting identity. Authorization Code with PKCE for user delegation, Client Credentials for service-owned M2M, scoped and time-bounded tokens, verified discovery and revocation.

6. Error Handling - Whether errors give agents enough information to recover. RFC 9457 Problem Details, is_retriable, suggestions array, trace_id, doc_uri, semantic exit codes.

7. Tool Design - Quality of tool definitions across any framework (MCP, AI SDK, LangChain). Descriptions as prompts ("when to use" and "do not use for"), typed schemas, toModelOutput, cross-framework portability.

8. Context Files - Quality of agent context files for AI coding assistants. A curated AGENTS.md as the baseline, with tool-specific overlays (CLAUDE.md, .cursor/rules) added only where a tool needs something different. Hand-curated, commands first, permission boundaries, progressive disclosure.

9. Testing & Evaluation - Whether agent interactions are tested and evaluated. Tool routing accuracy, error recovery testing, multi-step flow tests, pass@k metrics, eval-driven development.

10. Retrievability - Whether an agent can query your product's own data through a documented search or retrieval capability and trust what comes back. Search and retrieval endpoints, filters and pagination scoped to retrieval, freshness and provenance metadata on results.

Scored Dimensions vs Reference Sections

Not every section of Agent Surface is a scored dimension. The ten dimensions map onto the agent-readable-surface sections a codebase can be measured against; the remaining sections are reference and conceptual material that inform the score without being scored themselves.

The scorecard is product-side: it measures a product's own agent-facing surfaces, not how a team builds its own agents. The following sections are reference, conceptual, or part of the secondary agent-building inventory - they inform how you design and audit agent systems but are not scored dimensions: Agents, Agentic UI, Agent Retrieval, Protocols, Reference Links, Tooling Catalog, and this Scoring section itself. If you are looking for a section in the score and cannot find it, it is almost certainly one of these.

Confidence Levels

Every score includes a confidence marker:

Low-confidence scores should be flagged for manual re-verification.

The N/A Trap

Not all dimensions apply to every project. Mark a dimension N/A only when it is genuinely inapplicable:

• CLI Design = N/A when your project is a web app with no CLI tool (but if you have any CLI at all, score it)
• Discovery & AEO = N/A when your project is a pure library with no web presence
• Retrievability = N/A when your project exposes no searchable knowledge, documents, or user data to agents

"We haven't built an MCP server yet" is a 0, not N/A. "This project has no reason to expose an MCP server" might be N/A - but only if you are certain agents will never consume it.

Evidence Requirements

Every score must cite specific evidence:

• File paths with line numbers: openapi.yaml:45 - description is 4 words
• Grep findings: No files match /isError|is_retriable/
• Glob results: 0 files found matching **/*.mcp.json
• Command output: mytool --help --json returns exit code 2 with no structured output

Scores without evidence are guesses. Guesses drift high due to optimism bias. Always ground scores in concrete findings from the codebase.

Overall Rating

Once all applicable dimensions are scored, the total is scaled to 0–30 and mapped to a human-readable rating:

What's Next

• Rubric - Full 0/1/2/3 criteria for each dimension
• Evidence - What counts as evidence and how to avoid optimism bias
• Scorecard Format - JSON schema the skill emits
• Delta Scoring - Computing improvements before/after transformation
• Clustering - Organizing findings into actionable clusters
• Calibration - Maintaining rubric consistency across raters and time
• Public Scanners - Reading external agent-readiness scanners alongside a Surface audit

Evaluate the whole product journey

Use Evaluating Product Journeys to connect these dimensions around a real task. Begin at the intended entry point, retrieve authoritative decision facts, follow the permission and action path, exercise recovery, and inspect the final state. This supplies evidence for the existing rubric and helps prioritize defects by their effect on the user.