Summary

Agents truncate context for efficiency and weight earlier content more heavily. Documentation structured for humans (background, explanation, then answer) fails for . Lead with the answer in the first 200 words. Use answer-first structure: direct answer, then data (tables, details), then explanation. This ensures agents extract accurate answers from early paragraphs.

Put the direct answer in the first 200 words
Use tables for structured data (, plan comparison)
Follow with detailed explanation and edge cases
Use clear headings to enable agent section targeting
Place code examples right after relevant text

retrieve and quote your documentation. The structure of that documentation - where the answer appears in the page, how headings are organized, how code examples are placed - determines whether the agent extracts an accurate answer or paraphrases something plausible but wrong.

Answer-First Structure

Put the conclusion or direct answer in the first 200 words of the page. Agents truncate context for efficiency; they weight earlier content more heavily than later content. If the answer to "what is the rate limit" appears in paragraph seven, some agents will answer from an earlier, less accurate paragraph.

The traditional SEO pattern of "introduce the topic, explain the background, eventually answer" is the wrong structure for AEO. Lead with the answer:

# Rate Limits

The API is rate limited to **1,000 requests per minute** per API key.
Exceeding this limit returns HTTP 429 with a `Retry-After` header.

## By Plan

| Plan       | Requests/minute | Burst allowance |
| ---------- | --------------- | --------------- |
| Free       | 100             | 200 for 10s     |
| Pro        | 1,000           | 5,000 for 10s   |
| Enterprise | Custom          | Configurable    |

## Handling Rate Limit Errors

...

Contrast with the wrong structure:

# Rate Limits

Rate limiting is a common technique used by APIs to ensure fair usage
across all customers and to protect the stability of the platform.
Our rate limiting system uses a token bucket algorithm...

[four paragraphs later]

The default limit is 1,000 requests per minute.

Self-Contained Sections

Each H2 section should work as a standalone answer without requiring the reader (or agent) to have read the preceding sections. This is harder to write but dramatically increases retrieval accuracy.

When an agent receives a question about "how to handle signature verification," it may retrieve only the relevant section of your webhooks page - not the full page. If that section refers to "the keys you obtained earlier" without specifying what they are or where to find them, the agent produces an incomplete answer.

Write each H2 as if it were the first thing the reader sees:

## Verifying Webhook Signatures

All webhook requests include an `X-Webhook-Signature` header. Verify
this header to confirm the request originated from our servers.

The signature is an HMAC-SHA256 of the raw request body, using your
webhook secret as the key. Find your webhook secret in Settings →
Webhooks → [your endpoint] → Secret.

```typescript
import { createHmac } from "crypto";

function verifySignature(
  payload: string, // raw request body as string
  signature: string, // value of X-Webhook-Signature header
  secret: string, // your webhook secret from the dashboard
): boolean {
  const expected = createHmac("sha256", secret).update(payload).digest("hex");

  return `sha256=${expected}` === signature;
}
```

This section is complete. An agent can retrieve it and answer "how do I verify webhook signatures" without needing the surrounding page.

Canonical URL and Language

Two page-level signals sit underneath everything else on this page, and both are checked by name in Lighthouse's SEO and accessibility audits:

<link rel="canonical" href="https://docs.example.com/authentication" /> in <head> - Lighthouse's canonical audit fails a page without one. When the same content is reachable at more than one URL (tracking parameters, trailing slashes, a staging mirror), the canonical link tells an agent which one to cite and revisit.
<html lang="en"> on the root element - Lighthouse's html-has-lang audit fails a page without it. Ora's metadata-completeness check groups both together: "The basics AIs use to identify a page: canonical URL, language, and Open Graph image and type. Missing pieces mean mangled citations and wrong previews" (ora.ai/api/checks).

Heading Hierarchy

Never skip heading levels. H1 → H3 without an H2 confuses the document outline that agents use for navigation and retrieval.

H1 - Page title. One per page. Match the `<title>` tag.

H2 - Major topics. These are your retrieval units. Write H2 headings as questions or as direct answers to questions:

## How do I rotate an API key? ← question form (good for FAQPage schema)

## Rotating API Keys ← noun form (fine, clear enough)

## Important Information About Keys ← vague (avoid)

H3 - Subtopics within an H2. Do not use H3 for things that could be list items.

H4+ - Rarely necessary. If you are reaching H4, consider whether you should restructure as separate pages.

Code Examples Immediately After Prose

Place code examples directly after the prose that introduces them, rather than in an "Examples" section at the bottom of the page. Agents frequently retrieve a code example and the sentence immediately before it; if the code and its explanation are separated, the code appears without context.

## Paginating Results

Results are paginated using cursor-based pagination. Each response
includes a `next_cursor` field. Pass this value as the `cursor`
parameter in the next request.

```typescript
async function* getAllProjects(apiKey: string) {
  let cursor: string | undefined;

  do {
    const response = await fetch(
      `https://api.example.com/v2/projects${cursor ? `?cursor=${cursor}` : ""}`,
      { headers: { Authorization: `Bearer ${apiKey}` } },
    );

    const data = await response.json();
    yield* data.projects;
    cursor = data.next_cursor;
  } while (cursor);
}
```

When next_cursor is absent or null, you have reached the last page.

Always tag the fence with a language (```typescript, not a bare ```). CommonMark defines this as the fence's info string, whose first word is "typically used to specify the language of the code sample." An untagged fence forces an agent to guess the language before it can reason about the code correctly.

Note: closing text after the code block ("When next_cursor is absent...") re-anchors the code in the reader's understanding. Include it.

Tables Over Prose for Parameters

Parameter documentation in prose form is hard for agents to parse into structured data. Tables are unambiguous:

## Request Parameters

| Parameter     | Type    | Required | Description                                                     |
| ------------- | ------- | -------- | --------------------------------------------------------------- |
| `project_id`  | string  | Yes      | ID of the project to deploy                                     |
| `ref`         | string  | Yes      | Git ref (branch name, tag, or commit SHA)                       |
| `environment` | string  | No       | Target environment. Default: `production`                       |
| `strategy`    | string  | No       | Deployment strategy: `rolling` or `replace`. Default: `rolling` |
| `timeout`     | integer | No       | Maximum deployment duration in seconds. Default: `300`          |

Use the same format for response fields, environment variables, and configuration options.

Token Budgets

Keep page counts within these limits. Agents working with large (200K to 1M+ tokens) may load many pages simultaneously; pages that consume too many tokens limit how much can be loaded.

Page type	Target	Hard limit
Quick start / getting started	< 15K tokens	20K tokens
Concept explanation	< 10K tokens	15K tokens
reference (single endpoint)	< 5K tokens	8K tokens
API reference (full section)	< 25K tokens	30K tokens
Changelog / release notes	No limit	No limit

Pages over 30K tokens should be split. A "REST API Reference" page with every endpoint documented inline should become individual pages per endpoint group.

Estimate token count: character_count / 4 gives a rough approximation for English prose. Code-heavy content is slightly more token-dense.

Error Documentation

Error messages are high-value retrieval targets. When a user pastes an error into an AI chat, the AI searches for documentation matching that exact error text. Document errors with the exact message text as a heading or inline code:

## Error Reference

### `DEPLOYMENT_FAILED: timeout exceeded`

The deployment did not complete within the configured timeout.

**Causes:**

- Health check endpoint did not respond within the timeout period
- Container startup time exceeded the timeout

**Resolution:**
Increase the `timeout` parameter to at least 2x your container's typical startup time.
Check `/health` endpoint availability independently with `curl -f https://your-app.example.com/health`.

### `INVALID_REF: git ref not found`

The `ref` parameter does not exist in the linked repository.

**Causes:**

- Branch or tag was deleted after the deployment was initiated
- Typo in the `ref` parameter

**Resolution:**
Verify the ref exists: `` `git ls-remote origin | grep <ref>` ``.

Quote the exact error message string - including punctuation and capitalization - as it appears in API responses. Agents match against exact strings.

Meta Tags for Retrieval Accuracy

<head>
  <!-- Entity-first: product name before page topic -->
  <meta property="og:title" content="Acme API - Rate Limits" />
  <meta
    property="og:description"
    content="Rate limit thresholds by plan, handling 429 errors, and configuring retry logic for the Acme REST API."
  />

  <!-- dateModified is used by agents to assess freshness -->
  <meta property="article:modified_time" content="2025-03-15T09:00:00Z" />
  <meta property="article:published_time" content="2024-09-01T00:00:00Z" />

  <!-- Tags help retrieval systems categorize the page -->
  <meta property="article:tag" content="rate-limiting" />
  <meta property="article:tag" content="api" />
  <meta property="article:tag" content="errors" />
</head>

og:title entity-first - Put the product or service name first, then the topic. "Acme API - Rate Limits" retrieves more accurately than "Rate Limits" because the product context disambiguates the topic from other APIs' rate limit documentation.

article:modified_time - Agents and search-augmented systems use modification time to assess whether documentation is current. A page with no modification time is treated as unknown age. Use W3C datetime format: 2025-03-15T09:00:00Z.

article:tag - Retrieval systems use tags for categorization and filtering. Keep tags specific and consistent across your documentation.

Sitemap Optimization

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://docs.example.com/api/authentication</loc>
    <lastmod>2025-03-15</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.9</priority>
  </url>
  <url>
    <loc>https://docs.example.com/api/rate-limits</loc>
    <lastmod>2025-01-20</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

Search engines and AI crawlers use lastmod to prioritize recrawling. A sitemap without lastmod gives crawlers no signal about what has changed - they recrawl everything with equal priority. Use W3C datetime format: YYYY-MM-DD or the full ISO 8601 form.

Sitemap limits: maximum 50,000 URLs per sitemap file. For larger sites, use a sitemap index file that references multiple sitemaps.

Sitemap index:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://docs.example.com/sitemap-api.xml</loc>
    <lastmod>2025-03-15</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://docs.example.com/sitemap-guides.xml</loc>
    <lastmod>2025-03-10</lastmod>
  </sitemap>
</sitemapindex>

Submit your sitemap in robots.txt:

Sitemap: https://docs.example.com/sitemap.xml

Changelog and Feed Signals

lastmod tells an agent that something on a URL changed. A changelog tells it what changed, in language it can act on without re-reading the whole page. Publish a dated changelog - either a page with one entry per date and version, or a machine-parseable feed - so an agent that already has your docs in context can detect drift with one small fetch instead of re-ingesting everything.

RSS 2.0 and Atom (RFC 4287) both work as the feed format; Atom's updated element is unambiguous ISO 8601, which makes it the easier of the two to parse reliably. Advertise the feed the same way you'd advertise a alternate:

<link rel="alternate" type="application/atom+xml" href="/changelog/feed.xml" title="Changelog" />

Each entry needs a real date, a stable identifier, and text that says what changed and why it matters to someone calling your API or reading your docs - "Bumped dependencies" is not a useful entry; "Rate limit raised from 100 to 1,000 requests/minute on the Pro plan" is. There's no scanner check for this - it's a practice with real primary-source backing (the RSS and Atom specs) rather than a graded signal, but it's the cheapest way to let an agent trust that it isn't working from stale facts.

Semantic HTML

Search-augmented agents parse HTML. Semantic structure helps them identify the main content, skip navigation, and understand document hierarchy.

Use:

<main>
  <article>
    <!-- main content area -->
    <header>
      <h1>Rate Limits</h1>
      <time datetime="2025-03-15">March 15, 2025</time>
    </header>
    <section>
      <h2>Default Limits</h2>
      <p>...</p>
    </section>
  </article>
</main>

<nav aria-label="Documentation navigation">
  <!-- navigation links -->
</nav>

Avoid:

<div class="content">
  <div class="heading-1">Rate Limits</div>
  <div class="section">
    <div class="heading-2">Default Limits</div>
    <div class="body-text">...</div>
  </div>
</div>

Both render identically in a browser. To a parser or agent reading the HTML structure, the first is unambiguous; the second requires CSS class analysis to infer hierarchy.

A single <main> region matters beyond the landmark itself: it is what lets an agent (or a screen reader) separate the content it came for from navigation chrome, promo banners, and related-articles rails. Ora's ax-document-structure check reads this the same way a screen reader would: "Agents navigate your page the way screen readers do: by landmarks and headings. We check your HTML is a real document, with a main region, nav, and a sane heading order" (ora.ai/api/checks). Keep the actual content inside <main> large relative to the surrounding chrome - a page that is mostly navigation links, a cookie banner, and a footer around a few sentences of content gives a retrieval agent little to extract regardless of how clean the markup is.

Avoid Redirect Chains

Serve the final content URL directly rather than routing through a redirect chain, a meta-refresh stub, or a client-side window.location hop. Ora's redirect-hygiene check states the requirement directly: "Pages reach real content without meta-refresh stubs, JavaScript-only redirects, or cross-domain hops that strand non-JS agents" (ora.ai/api/checks). Most agent fetchers don't execute JavaScript, so a client-side redirect isn't slow for them - it's a dead end. Lighthouse's redirects audit flags the same pattern from a page-speed angle: each hop in the chain adds latency before any content is reachable at all.

Content Structure for AI Consumption