---
title: "Discovery & AEO"
description: "Making your capabilities findable by AI agents through structured discovery mechanisms"
url: "https://agentsurface.dev/docs/discovery"
lastVerified: 2026-09-24
lastModified: 2026-09-25T09:21:34.000Z
---



## Summary [#summary]

Discovery covers public, web-facing signals that help agents find, read, cite, and govern access to your docs, APIs, and capability metadata. Use it for sites, hosted documentation, public API catalogs, and crawler-facing policy.

* **Owns:** `llms.txt`, structured data, Markdown content negotiation, `robots.txt`, Content Signals, `.well-known` metadata, and public content structure.
* **Does not own:** repository/editor operating instructions; see [Context Files](/docs/context-files). MCP implementation details; see [MCP Servers](/docs/mcp-servers).
* **Agent-ready means:** agents can find canonical docs and machine-readable contracts, retrieve token-efficient content, and respect explicit access policy.

<Callout type="info">
  Agent-readiness is becoming an externally measured property. Cloudflare's [Agent Readiness
  score](https://isitagentready.com) (launched April 2026) grades any domain on many of the signals
  this section covers - `robots.txt`, Markdown content negotiation, Content Signals, Web Bot Auth,
  an RFC 9727 API Catalog, and MCP server cards. Treat it as a complementary independent checker to
  run alongside your own audit.
</Callout>

## Start Here [#start-here]

* [llms.txt](/docs/discovery/llms-txt) - Publish a compact index for agent retrieval.
* [AGENTS.md for Discovery](/docs/discovery/agents-md) - Expose public repository context where it belongs on the web.
* [Agent Skills](/docs/discovery/agent-skills) - Make reusable procedural capabilities findable.
* [Structured Data](/docs/discovery/structured-data) - Add JSON-LD and semantic page signals.
* [Content Negotiation](/docs/discovery/content-negotiation) - Serve clean Markdown to agents and HTML to browsers.
* [Well-Known Endpoints](/docs/discovery/well-known-endpoints) - Advertise API catalogs, MCP metadata, auth metadata, and keys.
* [DNS Discovery](/docs/discovery/dns-discovery) - Optionally advertise agent services to clients that support DNS-AID.
* [robots.txt](/docs/discovery/robots-txt) - Distinguish crawler access from permitted content use.
* [Content Structure](/docs/discovery/content-structure) - Write pages agents can retrieve and cite accurately.
* [Agent Discovery Checklist](/docs/discovery/aeo-checklist) - Audit the whole public discovery surface.

## Related [#related]

* [Context Files](/docs/context-files) - Repo-local instructions for coding agents.
* [API Surface](/docs/api-surface) - Machine-readable HTTP contracts linked from discovery pages.
* [MCP Servers](/docs/mcp-servers) - Servers that can be advertised through well-known metadata.
* [Authentication: Protected Resource Metadata](/docs/authentication/protected-resource-metadata) - OAuth metadata for protected APIs.
* [Reference Links](/docs/reference-links) - Standards and external sources behind the discovery guidance.

## Catalogs, commercial facts, and product use [#catalogs-commercial-facts-and-product-use]

* [Agent catalogs and structured feeds](/docs/discovery/catalogs-and-feeds) distinguishes ARD, AI Catalog, API Catalog, and NLWeb.
* [Commercial and entity discovery](/docs/discovery/commercial-and-entity-discovery) covers pricing.md, modular indexes, external identity, and distribution.
* [Evaluating Product Journeys](/docs/scoring/product-journeys) connects discovery to an informed decision, authorized action, and verified outcome.