Extract website metadata and Open Graph tags at the edge

Link previews, SEO audits, and agent crawls often need title, description, canonical, and Open Graph tags. Fetching HTML and walking it with HTMLRewriter on a Worker is fast and cheaper than launching Browser Rendering for every URL.

The Metadata Extractor experiment returns those fields as JSON with validated http(s) input.

Cloudflare products in this patternCopy Anchor Link

Fetch + rewrite vs headless ChromeCopy Anchor Link

Most marketing sites and blogs still ship og:* in the first response. Start cheap; escalate only when fields are missing. Official HTMLRewriter guide: HTMLRewriter · Workers.

What to extractCopy Anchor Link

A practical metadata payload:

• title - <title> or og:title
• description - meta description or og:description
• canonical - rel=canonical or og:url
• image - og:image
• siteName - og:site_name
• Optional: Twitter card tags, theme color, robots meta

Normalize absolute image URLs against the page origin. Truncate oversized descriptions for UI cards. Prefer UTF-8 decoding awareness for international sites.

HTMLRewriter sketchCopy Anchor Link

HTMLRewriter streams the response so you never buffer megabytes just to read a few tags:

rewriter.on('meta[property="og:title"]', {
  element(el) {
    title = el.getAttribute("content") ?? title;
  },
});

Handlers run as the HTML streams - see element handlers in the HTMLRewriter docs. The HTML Rewriter experiment isolates the API itself if you want a smaller learning surface.

Compose richer page APIsCopy Anchor Link

A link-preview product might call metadata first, then optionally screenshot when an image is missing.

Caching strategyCopy Anchor Link

Metadata for a URL is often stable for hours:

1. Build a cache key from canonical URL
2. caches.default.match (Cache API)
3. On MISS, fetch + rewrite, then put with s-maxage
4. For editorial previews, also store in KV with explicit purge on publish webhooks

Deploy and compareCopy Anchor Link

1. Read HTMLRewriter
2. Deploy Metadata Extractor
3. Point it at a blog post and a JS-heavy SPA
4. If SPA tags are empty, retry with Browser Rendering scrape / screenshot
5. Cache successful payloads

You get previews that stay cheap on the common path and only spend browser time when HTML is not enough - Workers as the vehicle, HTMLRewriter as the product capability.

Production checklistCopy Anchor Link

1. Validate URLs and block private destinations.
2. Prefer HTMLRewriter first; escalate to Browser Rendering only on miss.
3. Normalize absolute image URLs.
4. Cache successful payloads.
5. Respect robots/ToS for third-party targets.
6. Store preview cards with explicit TTLs.

FAQCopy Anchor Link

Why is og:image empty on SPAs?Copy Anchor Link

Client-rendered meta tags never appear in the first HTML. Use Browser Rendering or ask the site for SSR.

JSON-LD instead of OG?Copy Anchor Link

Many sites expose Product/Article entities in JSON-LD - see JSON-LD Extractor. Use both when building rich previews.