Summary

Computed diff between before and after scorecards (typically pre and post transformation) showing per-dimension improvements, overall score change, rating progression (e.g., Agent-tolerant → Agent-ready), and cluster impact identification. Use delta scorecards to track score changes over time and explain which fixes changed the result.

Per-dimension delta: Score change (1 → 2, +1 point)
Overall delta: Total score change (11/30 → 16/30, +5 points)
Rating delta: Band change (Agent-tolerant → Agent-ready)
Cluster impact: Which work clusters drove improvement
Trend tracking: Scorecard history for long-term progress
Stakeholder communication: Before/after visualization

A delta scorecard measures progress. By comparing before and after scorecards (typically before and after transformation), you can quantify which dimensions improved, which stayed flat, and which regressed.

Concept

A delta scorecard is a computed diff. It shows:

Score change per dimension (e.g., Surface: 1 → 2)
Overall score change (e.g., 11/30 → 16/30)
Rating change (e.g., Agent-tolerant → Agent-ready)
Which clusters were tackled and their impact

Use delta scorecards to:

Validate that transformation work actually improved scores
Identify which fixes had the most impact
Plan next priorities (address remaining high-severity issues)
Communicate progress to stakeholders

TypeScript Function

import { Scorecard } from "./scorecard-schema";

interface DeltaDimension {
  name: string;
  scoreBefore: number | null;
  scoreAfter: number | null;
  delta: number;
  confidenceBefore: string | null;
  confidenceAfter: string | null;
  improved: boolean;
  regressed: boolean;
}

interface DeltaScorecard {
  projectName: string;
  dateBefore: string;
  dateAfter: string;
  dimensions: DeltaDimension[];
  totalBefore: number;
  totalAfter: number;
  maxBefore: number;
  maxAfter: number;
  scaledBefore: number;
  scaledAfter: number;
  ratingBefore: string;
  ratingAfter: string;
  overallDelta: number;
  clustersApproached: string[];
}

/**
 * Compute delta between two scorecards.
 * Assumes both scorecards are from the same project.
 */
export function deltaScorecard(before: Scorecard, after: Scorecard): DeltaScorecard {
  // Validate project match
  if (before.project.name !== after.project.name) {
    throw new Error(`Project mismatch: ${before.project.name} vs ${after.project.name}`);
  }

  // Dimension deltas
  const dimensions: DeltaDimension[] = before.dimensions.map((dimBefore) => {
    const dimAfter = after.dimensions.find((d) => d.name === dimBefore.name);
    if (!dimAfter) {
      throw new Error(`Dimension ${dimBefore.name} not found in after scorecard`);
    }

    const scoreBefore = dimBefore.score;
    const scoreAfter = dimAfter.score;

    // Handle N/A dimensions
    const delta = scoreBefore === null || scoreAfter === null ? 0 : scoreAfter - scoreBefore;

    return {
      name: dimBefore.name,
      scoreBefore,
      scoreAfter,
      delta,
      confidenceBefore: dimBefore.confidence,
      confidenceAfter: dimAfter.confidence,
      improved: delta > 0,
      regressed: delta < 0,
    };
  });

  // Overall scores
  const totalBefore = before.overall.rawScore;
  const totalAfter = after.overall.rawScore;
  const maxBefore = before.overall.maxApplicable;
  const maxAfter = after.overall.maxApplicable;
  const overallDelta = totalAfter - totalBefore;

  // Cluster impact
  const clustersApproached = after.findingClusters
    .filter((cluster) =>
      cluster.findings.some((finding) => finding.currentScore < finding.targetScore),
    )
    .map((c) => c.name);

  return {
    projectName: before.project.name,
    dateBefore: before.date,
    dateAfter: after.date,
    dimensions,
    totalBefore,
    totalAfter,
    maxBefore,
    maxAfter,
    scaledBefore: before.overall.scaledScore,
    scaledAfter: after.overall.scaledScore,
    ratingBefore: before.overall.rating,
    ratingAfter: after.overall.rating,
    overallDelta,
    clustersApproached,
  };
}

/**
 * Format delta scorecard for display.
 */
export function formatDeltaScorecard(delta: DeltaScorecard): string {
  const lines: string[] = [
    "╔════════════════════════════════════════════════════════╗",
    "║              SURFACE DELTA SCORECARD                 ║",
    `║              ${delta.projectName.padEnd(47)}║`,
    "╠════════════════════════════════════════════════════════╣",
    "",
    "  Dimension                    Before  After   Delta",
    "  ─────────────────────────────────────────────────────",
  ];

  delta.dimensions.forEach((d) => {
    const before = d.scoreBefore === null ? "N/A" : `${d.scoreBefore}/3`;
    const after = d.scoreAfter === null ? "N/A" : `${d.scoreAfter}/3`;
    const deltaStr = d.delta > 0 ? `+${d.delta} ✦` : d.delta < 0 ? `${d.delta} ✗` : "  —";
    const status = d.improved ? "↑" : d.regressed ? "↓" : "→";
    lines.push(
      `  ${d.name.padEnd(30)}${before.padStart(6)}  ${after.padStart(5)}  ${deltaStr.padStart(6)} ${status}`,
    );
  });

  lines.push("");
  lines.push("  ─────────────────────────────────────────────────────");
  lines.push(
    `  TOTAL: ${delta.totalBefore}/${delta.maxBefore} → ${delta.totalAfter}/${delta.maxAfter} (scaled: ${delta.scaledBefore}/30 → ${delta.scaledAfter}/30)`,
  );
  lines.push(
    `  RATING: ${delta.ratingBefore} → ${delta.ratingAfter} ${delta.overallDelta > 0 ? "✦" : delta.overallDelta < 0 ? "✗" : "—"}`,
  );
  lines.push("");

  if (delta.clustersApproached.length > 0) {
    lines.push(`  Clusters tackled:`);
    delta.clustersApproached.forEach((c) => {
      lines.push(`    • ${c}`);
    });
    lines.push("");
  }

  lines.push("╚════════════════════════════════════════════════════════╝");

  return lines.join("\n");
}

Worked Example: Before & After

Before Scorecard

Project: acme-api audited on 2026-03-15

API Surface:        0/3 (no OpenAPI)
CLI Design:         N/A
MCP Server:         0/3 (no MCP)
Discovery & AEO:    1/3 (AGENTS.md only)
Authentication:     1/3 (API keys only)
Error Handling:     0/3 (generic HTTP)
Tool Design:        0/3 (no tool definitions)
Context Files:      1/3 (auto-generated AGENTS.md)
Testing:            0/3 (no agent tests)
Retrievability:     N/A

Total: 3/24 (scaled: 4/30)
Rating: Human-only

Transformation Work

Following the transformation plan, the team:

Created spec (2 hours)
- 40 endpoints with operationIds
- Agent-oriented descriptions (30–50 words each)
- Examples on all parameters
Implemented server (4 hours)
- Exposed 10 tools with proper annotations
- Structured error handling with isError
- Tested with InMemoryTransport
Upgraded authentication (3 hours)
- Added Client Credentials grant
- Scoped (read:api, write:config)
- validation (iss, aud, exp)
- Tested token renewal, revocation, and wrong-user/wrong-resource rejection
Enhanced error handling (1.5 hours)
- Problem Details format
- is_retriable on all errors
- suggestions array with recovery steps
Improved tool descriptions (1 hour)
- "Use when" context on all tools
- "Do not use for" disambiguation
- Parameter examples
Wrote agent tests (2 hours)
- Tool selection tests
- Error recovery tests
- Multi-step flow tests

After Scorecard

Project: acme-api audited on 2026-04-17

API Surface:        2/3 (OpenAPI with agent descriptions, no Arazzo)
CLI Design:         N/A
MCP Server:         2/3 (well-structured, no pagination)
Discovery & AEO:    2/3 (AGENTS.md + llms.txt, no JSON-LD)
Authentication:     3/3 (OAuth 2.1 M2M, scoped tokens)
Error Handling:     2/3 (RFC 9457, missing doc_uri)
Tool Design:        2/3 (good descriptions, no toModelOutput)
Context Files:      2/3 (AGENTS.md curated, not yet evolved from real friction)
Testing:            2/3 (tool + error recovery, no evals)
Retrievability:     N/A

Total: 17/24 (scaled: 21/30)
Rating: Agent-ready

Delta Computation

const before = JSON.parse(fs.readFileSync("scorecard-2026-03-15.json", "utf-8"));
const after = JSON.parse(fs.readFileSync("scorecard-2026-04-17.json", "utf-8"));

const delta = deltaScorecard(before, after);
console.log(formatDeltaScorecard(delta));

Output:

╔════════════════════════════════════════════════════════╗
║              SURFACE DELTA SCORECARD                 ║
║              acme-api                                  ║
╠════════════════════════════════════════════════════════╣

  Dimension                    Before  After   Delta
  ─────────────────────────────────────────────────────
  api-surface                   0/3    2/3   +2 ✦     ↑
  cli-design                     N/A    N/A    —      →
  mcp-server                     0/3    2/3   +2 ✦     ↑
  discovery-aeo                  1/3    2/3   +1 ✦     ↑
  authentication                 1/3    3/3   +2 ✦     ↑
  error-handling                 0/3    2/3   +2 ✦     ↑
  tool-design                    0/3    2/3   +2 ✦     ↑
  context-files                  1/3    2/3   +1 ✦     ↑
  testing                        0/3    2/3   +2 ✦     ↑
  retrievability                  N/A    N/A    —      →

  ─────────────────────────────────────────────────────
  TOTAL: 3/24 → 17/24 (scaled: 4/30 → 21/30)
  RATING: Human-only → Agent-ready ✦

  Clusters tackled:
    • Agent tool discoverability
    • Error recovery for agents
    • Machine-readable authentication

╚════════════════════════════════════════════════════════╝

Interpreting the Delta

Strong improvements (✦):

Authentication jumped from 1 → 3: OAuth 2.1 Client Credentials, scoped tokens, and tested renewal/revocation/wrong-resource rejection is complete
API Surface 0 → 2: OpenAPI spec exists with agent-oriented descriptions (missing for 3)
MCP Server 0 → 2: Tools exposed via MCP with proper structure (missing for 3)
Error Handling 0 → 2: RFC 9457 with is_retriable (missing doc_uri for 3)

Partial improvements (↑):

Discovery & 1 → 2: Added (missing and for 3)
Tool Design 0 → 2: Descriptions improved (missing toModelOutput optimization for 3)
Context Files 1 → 2: Curated (not yet evolved from real friction for 3)
Testing 0 → 2: Tool + error tests (missing metrics for 3)

Overall:

Rating upgraded: Human-only → Agent-ready (crossed the 15-point threshold)
Score improvement: +14 points in 13.5 hours of work
Impact: Agents can now reliably use the API with proper auth, error recovery, and tool discovery

Next Steps

From the delta, the team can prioritize remaining gaps:

API Surface 2 → 3: Add Arazzo workflows for multi-step operations (2 hours)
MCP Server 2 → 3: Implement pagination on list tools (1.5 hours)
Error Handling 2 → 3: Add doc_uri to error responses (1 hour)
Discovery 2 → 3: Add JSON-LD schema + content negotiation (2 hours)
Testing 2 → 3: Set up eval suite with metrics (3 hours)

Tackling these five 2→3 bumps would raise the raw score to 22/24, scaling to roughly 28/30 (Agent-first range).

Delta Scorecard