AI Website Summary

This is an experimental Worker. Use it as a starting point for your own projects.

The AI Website Summary experiment demonstrates how to fetch any webpage, extract its content, and generate a concise summary using Workers AI. Built with Hono and TypeScript, it runs entirely on Cloudflare's edge network.

Features

Fetch and parse any public webpage
Extract title and body text from HTML
Generate 2-4 sentence summaries using Llama 3.1
Sub-second response times on the edge
No external API keys required

API Reference

GET /summary

Generate an AI summary of any webpage.

Prop

Type

Response

title string | null

The extracted page title from the HTML <title> tag.

summary string

A 2-4 sentence AI-generated summary of the page content.

Example Request

curl "https://your-worker.workers.dev/summary?url=https://www.cloudflare.com"

Example Response

{
  "title": "Cloudflare - The Web Performance & Security Company",
  "summary": "Cloudflare provides a content delivery network and DDoS protection services. The platform offers security, performance, and reliability solutions for websites and applications. It includes features like CDN, DNS, and web application firewall to protect and accelerate online properties."
}

Error Responses

error string

Human-readable error message.

code string

Machine-readable error code.

400 Bad Request - Missing or invalid URL parameter:

{
  "error": "Missing or invalid query parameter: url",
  "code": "INVALID_URL"
}

502 Bad Gateway - Failed to fetch webpage or AI processing error:

{
  "error": "Failed to fetch or summarize",
  "code": "FETCH_OR_AI_ERROR"
}

Implementation Details

AI Model Configuration

The worker uses Workers AI with the following configuration:

// From src/constants/defaults.ts
export const AI_MODEL = "@cf/meta/llama-3.1-8b-instruct-fast";
export const MAX_SUMMARY_TOKENS = 4096;
export const MAX_HTML_BYTES = 128 * 1024; // 128KB limit
export const FETCH_TIMEOUT_MS = 15_000; // 15 second timeout

Request Flow

The summarization process follows these steps:

Validate URL

Ensures the URL uses http/https scheme and is properly formatted:

const url = validateUrl(c.req.query("url"));
if (!url) return jsonError(c, "Missing or invalid query parameter: url", "INVALID_URL");

Fetch HTML

Retrieves the webpage with a 15-second timeout and 128KB size limit:

const html = await fetchHtml(url);

Extract Content

Parses the HTML to extract title and body text:

const title = getTitle(html);
const bodyText = getBodyText(html, 6000); // Up to 6000 chars

Generate Summary

Sends content to Workers AI for summarization:

const summary = await summarizeWithAi(c.env, bodyText, title);

The AI prompt instructs the model to:

Generate exactly 2-4 concise sentences
Return only the summary (no preamble or labels)
Focus on key information from the page

Clean Response

Removes any boilerplate phrases the model may add:

// Strips patterns like "Here is a summary:" or "Let me know if..."
return extractSummaryOnly(response);

Core Implementation

Here's the main route handler from src/routes/summary.ts:

app.get("/summary", async (c) => {
  const url = validateUrl(c.req.query("url"));
  if (!url) return jsonError(c, "Missing or invalid query parameter: url", "INVALID_URL");

  try {
    const html = await fetchHtml(url);
    const title = getTitle(html);
    const bodyText = getBodyText(html, 6000);
    if (!bodyText.trim()) {
      return jsonSuccess<SummaryResponse>(c, { title, summary: "No extractable text content." });
    }
    const summary = await summarizeWithAi(c.env, bodyText, title);
    const response: SummaryResponse = { title, summary };
    return jsonSuccess(c, response);
  } catch (e) {
    const message = e instanceof Error ? e.message : "Failed to fetch or summarize";
    return jsonError(c, message, "FETCH_OR_AI_ERROR", 502);
  }
});

Use Cases

Content Curation - Automatically generate summaries for bookmarking tools
Research Assistants - Quickly preview webpage content before visiting
News Aggregators - Create summaries for article feeds
Browser Extensions - Add "summarize this page" functionality
Slack/Discord Bots - Share link summaries in team chats

Limitations

JavaScript-heavy sites: Only static HTML is fetched; JavaScript-rendered content is not executed
Rate limits: Workers AI has usage limits based on your plan
Fetch timeout: Requests that take longer than 15 seconds will fail
Size limit: HTML content is capped at 128KB; body text is truncated to 6000 characters

Deployment

Click the deploy button

Deploy

Enable the Workers AI binding (AI) in your Worker settings. The deploy button configures this automatically via wrangler.json. Requires a Cloudflare account with Workers AI enabled.

Test your deployment

curl "https://your-worker.workers.dev/summary?url=https://www.cloudflare.com"

Local Development

cd apps/experiments/ai-website-summary
npm install
npm run dev

Test locally:

curl "http://localhost:8787/summary?url=https://www.cloudflare.com"

Configuration

The Worker automatically binds to Workers AI. The wrangler.json configuration includes:

{
  "name": "ai-website-summary",
  "main": "src/index.ts",
  "compatibility_date": "2024-01-01",
  "ai": { "binding": "AI" }
}

No additional environment variables or secrets are required.

Dependencies

{
  "dependencies": {
    "hono": "^4.6.12"
  },
  "devDependencies": {
    "@cloudflare/workers-types": "^4.20241127.0",
    "typescript": "^5.7.2",
    "wrangler": "^4"
  }
}

Cloudflare Features Used

Workers - Serverless execution environment
Workers AI - Run LLMs on the edge with the AI binding
Fetch API - HTTP client for fetching webpages

Next Steps

Workers AI Models

Explore other models available in Workers AI

Hono Framework

Learn more about the Hono web framework

GitHub Repository

View the complete source code

More Experiments

Explore other Cloudflare experiments

AI Website Summary

Workers AI Models

Hono Framework

GitHub Repository

More Experiments

On this page