AI Website Tag Generator

The AI Website Tag Generator experiment fetches any webpage, extracts its content, and uses Workers AI to generate topic tags. Built with Hono and TypeScript, it runs on Cloudflare's edge network-no external API keys required.

FeaturesCopy Anchor Link

• Fetch and parse any public webpage
• Extract title and body text from HTML
• Generate topic tags using Workers AI (e.g. Llama 3.1)
• Sub-second responses on the edge
• Accepts full URLs or hostnames (e.g. https://cloudflare.com or example.com)

API ReferenceCopy Anchor Link

GET /tagsCopy Anchor Link

Generate topic tags for any webpage.

ResponseCopy Anchor Link

tags string[]

An array of topic tags generated from the page content.

Example RequestCopy Anchor Link

curl "https://your-worker.workers.dev/tags?url=https://www.cloudflare.com"

Example ResponseCopy Anchor Link

{
  "tags": ["technology", "CDN", "security", "edge computing", "DDoS protection"]
}

Error ResponsesCopy Anchor Link

error string

Human-readable error message.

code string

Machine-readable error code.

400 Bad Request - Missing or invalid URL:

{
  "error": "Missing or invalid query parameter: url",
  "code": "INVALID_URL"
}

502 Bad Gateway - Failed to fetch webpage or AI error:

{
  "error": "Failed to fetch or generate tags",
  "code": "FETCH_OR_AI_ERROR"
}

Use CasesCopy Anchor Link

• Content tagging - Auto-tag articles or bookmarks by topic
• SEO and discovery - Suggest categories or related topics from page content
• Link previews - Enrich shared links with topic tags
• Research and curation - Quickly label pages by theme

LimitationsCopy Anchor Link

• JavaScript-heavy sites - Only static HTML is fetched; client-rendered content is not executed
• Workers AI limits - Subject to Workers AI usage limits by plan
• Fetch and size limits - Same constraints as other URL-based experiments (timeout, HTML size cap)

Use in your projectCopy Anchor Link

import type { Env } from "../types/env";import { AI_MODEL, MAX_TAG_TOKENS, MAX_TAGS } from "../constants/defaults";/** * Parse AI response into lowercase tags: split on newlines and commas, trim, dedupe, cap. */function parseTags(raw: string): string[] {  const seen = new Set<string>();  const tags: string[] = [];  const parts = raw    .split(/[\n,]+/)    .map((s) =>      s        .replace(/^[\d.)\s\-*]+/, "")        .trim()        .toLowerCase()    )    .filter(Boolean);  for (const tag of parts) {    const normalized = tag.replace(/\s+/g, "-").slice(0, 50);    if (normalized && !seen.has(normalized) && tags.length < MAX_TAGS) {      seen.add(normalized);      tags.push(normalized);    }  }  return tags;}export async function generateTagsWithAi(  env: Env,  text: string,  title: string | null): Promise<string[]> {  const prompt = `You are a tag generator. Based on the following webpage content, output 3 to 10 short topic tags. Use single words or short hyphenated phrases (e.g. "machine-learning"). Output ONLY the tags: one per line or comma-separated. No numbering, no preamble, no "Tags:" label, no explanation.${title ? `Page title: ${title}\n\n` : ""}Content:\n${text}`;  const out = await env.AI.run(AI_MODEL, {    prompt,    max_tokens: MAX_TAG_TOKENS,  });  const response =    (out as { response?: string }).response ?? (out as { result?: string }).result ?? "";  const tags = parseTags(response);  return tags.length > 0 ? tags : ["uncategorized"];}

DeploymentCopy Anchor Link

curl "https://your-worker.workers.dev/tags?url=https://www.cloudflare.com"

Local DevelopmentCopy Anchor Link

cd apps/experiments/ai-website-tag-generator
npm install
npm run dev

Test locally:

curl "http://localhost:8787/tags?url=https://www.cloudflare.com"

Cloudflare Features UsedCopy Anchor Link

• Workers - Serverless execution environment
• Workers AI - Run LLMs on the edge with the AI binding
• Fetch API - HTTP client for fetching webpages

Next StepsCopy Anchor Link