Discovery & AEO
Agent Catalogs and Structured Feeds
Distinguish ARD, AI Catalog, API Catalog, and NLWeb feeds and verify the resources they advertise
Last verified 2026-09-24
Catalogs help a client locate a capability. They do not implement that capability, grant permission, or prove a publisher trustworthy. Choose a format because a consumer uses it, then validate both the document and the advertised target.
ARD and AI Catalog
Agentic Resource Discovery publishes entries at /.well-known/ard.json and formally defines three ways to advertise that location: an Agentmap: directive in robots.txt, an HTML <link rel="ard"> tag, and DNS service binding (ARD spec §5.1). Its current specification keeps the predecessor ai-catalog.json path and the rel="ai-catalog" link relation as an optional fallback a consumer "MAY additionally consult" - neither is part of ARD's own discovery contract. AI Catalog is the separate, earlier specification that actually owns that path and relation, at /.well-known/ai-catalog.json.
Lighthouse's ard-schema audit checks a fixed sequence and stops at the first hit: the Agentmap: directive in robots.txt, then an HTML link[rel="ai-catalog"] tag, then an HTTP Link header with rel="ai-catalog", then /.well-known/ai-catalog.json as the final fallback (source). That order mixes ARD's own Agentmap mechanism with AI Catalog's predecessor rel and path. A pass on this audit confirms a consumer can locate some catalog document - it does not confirm you have implemented ARD's current advertisement contract, which uses rel="ard" rather than rel="ai-catalog". Publish rel="ard" for ARD-native consumers, and add the ai-catalog path or rel only if you also target the predecessor format.
Do not assume the two formats will remain interchangeable. ARD now defines its own entry schema. Validate each advertised representation against the exact schema revision you target. ARD schema and discovery rules.
For each resource, verify publisher identity, identifier, display name, media type, and the choice between a linked resource and inline data. Resolve links and invoke the actual protocol with a compatible client. An document, endpoint, skill, and card have different contracts even when listed together.
A trustManifest is a claim envelope. Check its declared verification framework, issuer, expiry, subject, and publisher binding. A signature-shaped string or unsupported compliance claim is not evidence. Record unverifiable claims as unverified; publish only attestations that a consumer can substantiate.
DNS discovery is another route
Clients may also look up agent endpoints through DNS discovery. is an emerging proposal with its own service records. It complements an HTTP catalog; neither route establishes that a discovered service is authorized or usable.
Cloudflare's scanner also probes _catalog._agents TXT and _search._agents SRV records as part of its catalog check. These are observed discovery routes, not evidence that every ARD consumer requires them. Use the current specification and your client's supported discovery mechanism when deciding what to publish.
API Catalog is another surface
The RFC 9727 API Catalog at /.well-known/api-catalog inventories through linksets. It neither replaces ARD nor turns an HTML developer page into an API contract. Publish the formats your intended clients can consume and keep all of them derived from the same capability inventory.
NLWeb feeds and query endpoints
NLWeb's reference implementation uses structured content and natural-language queries. It is a server-side retrieval surface; WebMCP exposes tools within a browser page. They serve different execution contexts.
Design and verify three independent parts of the retrieval surface:
| Signal | Implementation and evidence |
|---|---|
| Schema feeds | Advertise structured feeds through the discovery mechanism supported by the consumer. For integrations using a schemamap: directive and Schema Map XML, validate against their specified revision. Download and parse listed feeds, including changed and removed records. |
/ask | Use the NLWeb request and response contract for the pinned implementation, with source URLs and structured result metadata. A generic chatbot response or a route named /ask is insufficient. |
| Streaming | Exercise a streaming request and inspect event framing, completion, cancellation, and errors. An HTTP 200 or the SSE content type alone does not demonstrate a usable stream. |
The reference REST API documentation describes a query argument and a streaming option. Pin the implementation and contract your client uses, and retain an actual request and response as compatibility evidence.
Keep ingestion , record identity, and deletion behavior explicit. The feed and query contracts must agree on what each record represents and which source owns it. Choose these integrations when structured retrieval serves a real consumer need.
Verification record
Save the fetched document, schema revision, response headers, target URLs, validation output, and one real invocation. Include authentication requirements and retrieval time. Recheck after changing hosts, schemas, or capability names. Connect that evidence to an actual product journey: finding a resource, understanding its requirements, and successfully using it.