How Joost de Valk transformed joost.blog into a Level 5 agent-native site using content negotiation, MCP servers, WebMCP tools, and well-known URIs — all on Cloudflare Pages with zero exotic infrastructure.
Joost de Valk ran his static blog through Cloudflare's isitagentready.com tool and scored 25 out of 100. The tool evaluates five categories: Discoverability, Content, Bot Access Control, API/Auth/MCP/Skill Discovery, and Commerce. A static Astro site with RSS, sitemaps, and schema.org markup was insufficient for the agent-native web. AI agents couldn't find machine-readable resources without parsing HTML — and the site had no API catalog, no MCP endpoint, no agent skills discovery, and no content negotiation.
Rather than a rewrite, the author addressed each failing check incrementally — adding Link headers for discoverability, pre-building markdown files for content negotiation, publishing well-known URIs for API and skill discovery, and implementing MCP/WebMCP tool endpoints. Each fix was verified by re-scanning with isitagentready.com, driving the score from 25 → 83.
Link headers pointing agents to sitemap, llms.txt, API catalog, agent skills index, MCP server card, and A2A agent card — machine-readable discovery without HTML parsing.
Markdown content negotiation via Accept: text/markdown headers — pre-built .md files with link rel=alternate tags and Cloudflare Transform Rules for path rewriting.
Content-Signal directive in robots.txt declaring AI training, search, and input permissions. Web Bot Auth request signing reserved for future adoption.
RFC 9727 API catalog, Agent Skills index, MCP server card, A2A agent card, WebMCP tools, and MCP server endpoint — the densest category covering all machine-readable discovery surfaces.
x402 HTTP 402 micropayment protocol for agent API access — marked not applicable for most non-commerce sites but flagged as inconsistent in Cloudflare's own scoring.
Machine-readable API listing at /.well-known/api-catalog — JSON describing available endpoints, schemas, and authentication.
Coding-agent skill loading via /.well-known/agent-skills/index.json — name, description, trigger phrases, and source URL per skill.
Server discovery at /.well-known/mcp/server-card.json — transport protocol, available tools, and auth requirements.
Agent-to-agent discovery at /.well-known/agent-card.json — enabling interoperable agent communication across platforms.
Browser-native navigator.modelContext.registerTool API — in-browser agent tool registration without a backend server.
AI training, search, and input preferences declared via Content-Signal header in robots.txt — 'ai-train=yes, search=yes, ai-input=yes'.
Machine-readable resource discovery from HTTP response headers — agents find resources without parsing HTML.
RAG-powered question-answering tool backed by Llama 3.3 70B via Cloudflare Workers AI. Uses keyword + semantic retrieval against the blog's ~27KB writing index, returning markdown citations parsed into a source list. Exposed via WebMCP and MCP.
Returns recent blog posts with URL, title, publish date, and excerpt. Provides a programmatic alternative to RSS and sitemaps — structured query access to the blog's content index.
Pre-build markdown files, API catalogs, and skill indices at compile time. Serving markdown means the file has to exist — Astro endpoints generate them during build, not at request time.
The entire agent-ready stack runs on Cloudflare Pages with one ~140-line Pages Function, build-time file generation, _headers rules, and a Transform Rule. No dedicated servers required.
Every agent resource must be discoverable via Link headers or well-known URIs — agents should never need to parse HTML to find sitemaps, API catalogs, or skill indices.
Intentionally public APIs should be expressible as such. The isitagentready tool penalizes unauthenticated endpoints because it can't express intentional openness.
Only expose tools that add value beyond existing standards — don't wrap RSS, sitemaps, or schema.org in MCP tools. The two exposed tools provide capabilities standards don't cover.
| # | Surface | Format | Purpose |
|---|---|---|---|
| 1 | robots.txt | Text | Crawler directives + Content-Signal header |
| 2 | sitemap.xml | XML | URL discovery for crawlers and agents |
| 3 | RSS feed | XML | Content syndication |
| 4 | llms.txt | Markdown | LLM-oriented site index |
| 5 | schema.org | JSON-LD | Structured data markup |
| 6 | Content-Signal header | HTTP header | AI training/search/input permissions |
| 7 | Link headers | HTTP header | Resource discovery without HTML parsing |
| 8 | Markdown (content negotiation) | text/markdown | Machine-readable article content |
| 9 | API catalog (RFC 9727) | JSON | Endpoint and schema discovery |
| 10 | Agent Skills index | JSON | 8 skills for AI coding agents |
| 11 | MCP server card | JSON | MCP endpoint and tool discovery |
| 12 | A2A agent card | JSON | Agent-to-agent interoperability |
| 13 | WebMCP tools | JavaScript API | In-browser agent tools |
| 14 | MCP server endpoint | JSON-RPC | Server-side tool execution |
More than tripled the agent-readiness score through incremental, category-by-category improvements verified by isitagentready.com.
Attained the highest tier — Agent-Native. All five scoring categories addressed with production-quality implementations.
From robots.txt to MCP endpoint — 14 distinct surfaces available for AI agent interaction, each discoverable without HTML parsing.
Skills published on GitHub and indexed at /.well-known/agent-skills/index.json — loadable by Claude Code, Cursor, and GitHub Copilot.
Create a _headers file pointing agents to sitemap.xml, llms.txt, API catalog, skills index, MCP server card, and A2A card. Agents find every resource from a single HTTP response.
Use static site endpoint routes to generate .md files during build. Add link rel=alternate type=text/markdown tags. The file must exist on disk for content negotiation.
Set up a Cloudflare Transform Rule to rewrite paths to .md files when Accept includes text/markdown. Same URL, machine-readable format.
Deploy API catalog, Agent Skills index, MCP server card, and A2A agent card as static JSON files at /.well-known/ — all built at compile time.
Use navigator.modelContext.registerTool to expose tools in-browser. Register only tools adding value beyond RSS, sitemaps, and schema.org.
Build a ~140-line Cloudflare Pages Function using Streamable HTTP transport (JSON-RPC over POST) for external MCP clients like Claude Desktop.
Add Content-Signal: ai-train=yes, search=yes, ai-input=yes to robots.txt. Reserve Web Bot Auth signing for when widely adopted.
Agent readiness measures how well a website accommodates AI agent consumption — discoverability of structured data, content negotiation for machine-readable formats, documented APIs, and tool endpoints. As AI agents become primary consumers of web content, sites that aren't agent-ready become invisible to AI-assisted workflows.
Cloudflare's free site-scoring tool that evaluates websites across five categories — Discoverability, Content, Bot Access Control, API/Auth/MCP/Skill Discovery, and Commerce — producing a 0-100 score. The author used it iteratively to identify gaps and verify fixes.
Pre-build .md files at compile time using Astro endpoints, add link rel=alternate type=text/markdown tags, and configure Cloudflare Transform Rules to rewrite paths when requests carry Accept: text/markdown headers. The key constraint: the .md file must physically exist on disk.
/.well-known/api-catalog (RFC 9727), /.well-known/agent-skills/index.json (Agent Skills RFC), /.well-known/mcp/server-card.json (SEP-1649), and /.well-known/agent-card.json (A2A Protocol). These let agents discover capabilities without parsing HTML.
WebMCP is browser-native (navigator.modelContext.registerTool) for in-browser tool registration — no backend required. MCP is server-side, using Streamable HTTP transport (JSON-RPC over POST) for external agent clients. The author implements both for the same two tools.
Eight skills hosted on GitHub: Astro SEO configuration, GitHub repo setup, WordPress readme optimization, readability checking, and others. Each skill is described in the Agent Skills index with name, description, trigger phrases, and source URL.
A RAG-powered Q&A tool using keyword + semantic retrieval against the blog's ~27KB writing index, generating answers with Llama 3.3 70B via Cloudflare Workers AI, and returning markdown citations parsed into a source list. Exposed via WebMCP and MCP.
Joost's APIs are intentionally public and unauthenticated. The isitagentready scoring tool lacks a way to express 'this API is intentionally public,' so the check fails. The author argues the tool should distinguish between missing auth and deliberately open endpoints.
None exotic — joost.blog runs entirely on Cloudflare Pages with a single ~140-line Pages Function, build-time static file generation via Astro endpoints, a _headers file, and one Transform Rule. No dedicated servers, databases, or Kubernetes required.
An IETF draft spec declaring AI training, search, and input preferences via a Content-Signal header in robots.txt. The author uses 'ai-train=yes, search=yes, ai-input=yes' to explicitly permit AI agents to train on, search, and use his content as input.
x402 is an HTTP 402 micropayment protocol enabling agent API access with stablecoin micropayments. The author marks it not applicable for most non-commerce sites but flags an inconsistency: Cloudflare's own EmDash CMS supports x402, yet the scoring tool categorizes it as a commerce edge case.
Score improved from 25/100 to 83/100, achieving Level 5 (Agent-Native). The site now exposes 14 distinct agent interaction surfaces. Eight agent skills published. Two MCP/WebMCP tools operational. All on a static Cloudflare Pages site with no exotic infrastructure.
A measure of how well a website accommodates AI agent consumption — scored 0-100 across five categories.
Cloudflare's scoring tool evaluating sites on discoverability, content, bot access, API/MCP/skill discovery, and commerce.
Serving markdown when agents request Accept: text/markdown — enabled by pre-built .md files and Transform Rules.
RFC proposing /.well-known/agent-skills/index.json for coding-agent skill loading by Claude Code, Cursor, and Copilot.
Server-side protocol for exposing tools to AI agents via Streamable HTTP transport (JSON-RPC over POST).
Browser-native navigator.modelContext.registerTool API for in-browser tool registration — no backend required.
Google-standard for agent-to-agent discovery via Agent Card at /.well-known/agent-card.json.
IETF draft declaring AI training/search/input permissions via Content-Signal header in robots.txt.
IETF standard for JSON-based API catalog at /.well-known/api-catalog for machine-readable endpoint discovery.
Draft spec for MCP server discovery — JSON card at /.well-known/mcp/server-card.json.
HTTP 402 Payment Required protocol for agent API access with stablecoin micropayments.
URL rewriting rules that modify request paths based on headers — used for markdown content negotiation.