Agent Surface

Make your product agent-ready

Discovery & AEO

Making your capabilities findable by AI agents through structured discovery mechanisms

Last verified 2026-09-24

Summary

Discovery covers public, web-facing signals that help agents find, read, cite, and govern access to your docs, , and capability metadata. Use it for sites, hosted documentation, public API catalogs, and crawler-facing policy.

  • Owns: llms.txt, structured data, , robots.txt, , .well-known metadata, and public content structure.
  • Does not own: repository/editor operating instructions; see Context Files. implementation details; see MCP Servers.
  • Agent-ready means: agents can find canonical docs and machine-readable contracts, retrieve token-efficient content, and respect explicit access policy.

Agent-readiness is becoming an externally measured property. Cloudflare's Agent Readiness score (launched April 2026) grades any domain on many of the signals this section covers - robots.txt, Markdown content negotiation, Content Signals, , an API Catalog, and MCP server cards. Treat it as a complementary independent checker to run alongside your own audit.

Start Here

  1. llms.txt - Publish a compact index for agent retrieval.
  2. AGENTS.md for Discovery - Expose public repository context where it belongs on the web.
  3. Agent Skills - Make reusable procedural capabilities findable.
  4. Structured Data - Add JSON-LD and semantic page signals.
  5. Content Negotiation - Serve clean Markdown to agents and HTML to browsers.
  6. Well-Known Endpoints - Advertise API catalogs, MCP metadata, auth metadata, and keys.
  7. DNS Discovery - Optionally advertise agent services to clients that support DNS-AID.
  8. robots.txt - Distinguish crawler access from permitted content use.
  9. Content Structure - Write pages agents can retrieve and cite accurately.
  10. Agent Discovery Checklist - Audit the whole public discovery surface.
  1. Context Files - Repo-local instructions for coding agents.
  2. API Surface - Machine-readable HTTP contracts linked from discovery pages.
  3. MCP Servers - Servers that can be advertised through well-known metadata.
  4. Authentication: Protected Resource Metadata - OAuth metadata for protected APIs.
  5. Reference Links - Standards and external sources behind the discovery guidance.

Catalogs, commercial facts, and product use

On this page

SummaryStart HereRelatedCatalogs, commercial facts, and product use