# AI Website Summary (/docs/experiments/ai-website-summary)



<Callout type="warning">
  This is an experimental Worker. Use it as a starting point for your own projects.
</Callout>

The AI Website Summary experiment demonstrates how to fetch any webpage, extract its content, and generate a concise summary using [Workers AI](https://developers.cloudflare.com/workers-ai/). Built with Hono and TypeScript, it runs entirely on Cloudflare's edge network.

## Features [#features]

* Fetch and parse any public webpage
* Extract title and body text from HTML
* Generate 2-4 sentence summaries using Llama 3.1
* Sub-second response times on the edge
* No external API keys required

## API Reference [#api-reference]

### GET /summary [#get-summary]

Generate an AI summary of any webpage.

<TypeTable
  type="{
  url: {
    description: &#x22;The webpage URL to summarize. Must use `http://` or `https://` scheme.&#x22;,
    type: &#x22;string&#x22;,
    required: true,
  },
}"
/>

#### Response [#response]

**`title`** `string | null`

The extracted page title from the HTML `<title>` tag.

**`summary`** `string`

A 2-4 sentence AI-generated summary of the page content.

#### Example Request [#example-request]

```bash
curl "https://your-worker.workers.dev/summary?url=https://www.cloudflare.com"
```

#### Example Response [#example-response]

```json
{
  "title": "Cloudflare - The Web Performance & Security Company",
  "summary": "Cloudflare provides a content delivery network and DDoS protection services. The platform offers security, performance, and reliability solutions for websites and applications. It includes features like CDN, DNS, and web application firewall to protect and accelerate online properties."
}
```

#### Error Responses [#error-responses]

**`error`** `string`

Human-readable error message.

**`code`** `string`

Machine-readable error code.

**400 Bad Request** - Missing or invalid URL parameter:

```json
{
  "error": "Missing or invalid query parameter: url",
  "code": "INVALID_URL"
}
```

**502 Bad Gateway** - Failed to fetch webpage or AI processing error:

```json
{
  "error": "Failed to fetch or summarize",
  "code": "FETCH_OR_AI_ERROR"
}
```

## Implementation Details [#implementation-details]

### AI Model Configuration [#ai-model-configuration]

The worker uses Workers AI with the following configuration:

```typescript
// From src/constants/defaults.ts
export const AI_MODEL = "@cf/meta/llama-3.1-8b-instruct-fast";
export const MAX_SUMMARY_TOKENS = 4096;
export const MAX_HTML_BYTES = 128 * 1024; // 128KB limit
export const FETCH_TIMEOUT_MS = 15_000; // 15 second timeout
```

### Request Flow [#request-flow]

The summarization process follows these steps:

<Steps>
  <Step>
    ### Validate URL [#validate-url]

    Ensures the URL uses http/https scheme and is properly formatted:

    ```typescript
    const url = validateUrl(c.req.query("url"));
    if (!url) return jsonError(c, "Missing or invalid query parameter: url", "INVALID_URL");
    ```
  </Step>

  <Step>
    ### Fetch HTML [#fetch-html]

    Retrieves the webpage with a 15-second timeout and 128KB size limit:

    ```typescript
    const html = await fetchHtml(url);
    ```
  </Step>

  <Step>
    ### Extract Content [#extract-content]

    Parses the HTML to extract title and body text:

    ```typescript
    const title = getTitle(html);
    const bodyText = getBodyText(html, 6000); // Up to 6000 chars
    ```
  </Step>

  <Step>
    ### Generate Summary [#generate-summary]

    Sends content to Workers AI for summarization:

    ```typescript
    const summary = await summarizeWithAi(c.env, bodyText, title);
    ```

    The AI prompt instructs the model to:

    * Generate exactly 2-4 concise sentences
    * Return only the summary (no preamble or labels)
    * Focus on key information from the page
  </Step>

  <Step>
    ### Clean Response [#clean-response]

    Removes any boilerplate phrases the model may add:

    ```typescript
    // Strips patterns like "Here is a summary:" or "Let me know if..."
    return extractSummaryOnly(response);
    ```
  </Step>
</Steps>

### Core Implementation [#core-implementation]

Here's the main route handler from `src/routes/summary.ts`:

```typescript
app.get("/summary", async (c) => {
  const url = validateUrl(c.req.query("url"));
  if (!url) return jsonError(c, "Missing or invalid query parameter: url", "INVALID_URL");

  try {
    const html = await fetchHtml(url);
    const title = getTitle(html);
    const bodyText = getBodyText(html, 6000);
    if (!bodyText.trim()) {
      return jsonSuccess<SummaryResponse>(c, { title, summary: "No extractable text content." });
    }
    const summary = await summarizeWithAi(c.env, bodyText, title);
    const response: SummaryResponse = { title, summary };
    return jsonSuccess(c, response);
  } catch (e) {
    const message = e instanceof Error ? e.message : "Failed to fetch or summarize";
    return jsonError(c, message, "FETCH_OR_AI_ERROR", 502);
  }
});
```

## Use Cases [#use-cases]

* **Content Curation** - Automatically generate summaries for bookmarking tools
* **Research Assistants** - Quickly preview webpage content before visiting
* **News Aggregators** - Create summaries for article feeds
* **Browser Extensions** - Add "summarize this page" functionality
* **Slack/Discord Bots** - Share link summaries in team chats

## Limitations [#limitations]

* **JavaScript-heavy sites**: Only static HTML is fetched; JavaScript-rendered content is not executed
* **Rate limits**: Workers AI has [usage limits](https://developers.cloudflare.com/workers-ai/platform/limits/) based on your plan
* **Fetch timeout**: Requests that take longer than 15 seconds will fail
* **Size limit**: HTML content is capped at 128KB; body text is truncated to 6000 characters

## Deployment [#deployment]

<Steps>
  <Step>
    ### Click the deploy button [#click-the-deploy-button]

    [![Deploy to Cloudflare Workers](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/shrinathsnayak/cloudflare-experiments/tree/main/apps/experiments/ai-website-summary)
  </Step>

  <Step>
    ### Deploy [#deploy]

    Enable the **Workers AI** binding (`AI`) in your Worker settings. The deploy button configures this automatically via `wrangler.json`. Requires a Cloudflare account with Workers AI enabled.
  </Step>

  <Step>
    ### Test your deployment [#test-your-deployment]

    ```bash
    curl "https://your-worker.workers.dev/summary?url=https://www.cloudflare.com"
    ```
  </Step>
</Steps>

## Local Development [#local-development]

```bash
cd apps/experiments/ai-website-summary
npm install
npm run dev
```

Test locally:

```bash
curl "http://localhost:8787/summary?url=https://www.cloudflare.com"
```

## Configuration [#configuration]

The Worker automatically binds to Workers AI. The `wrangler.json` configuration includes:

```json
{
  "name": "ai-website-summary",
  "main": "src/index.ts",
  "compatibility_date": "2024-01-01",
  "ai": { "binding": "AI" }
}
```

No additional environment variables or secrets are required.

### Dependencies [#dependencies]

```json
{
  "dependencies": {
    "hono": "^4.6.12"
  },
  "devDependencies": {
    "@cloudflare/workers-types": "^4.20241127.0",
    "typescript": "^5.7.2",
    "wrangler": "^4"
  }
}
```

## Cloudflare Features Used [#cloudflare-features-used]

* **[Workers](https://developers.cloudflare.com/workers/)** - Serverless execution environment
* **[Workers AI](https://developers.cloudflare.com/workers-ai/)** - Run LLMs on the edge with the `AI` binding
* **[Fetch API](https://developers.cloudflare.com/workers/runtime-apis/fetch/)** - HTTP client for fetching webpages

## Next Steps [#next-steps]

<Cards>
  <Card title="Workers AI Models" href="https://developers.cloudflare.com/workers-ai/models/">
    Explore other models available in Workers AI
  </Card>

  <Card title="Hono Framework" href="https://hono.dev">
    Learn more about the Hono web framework
  </Card>

  <Card title="GitHub Repository" href="https://github.com/shrinathsnayak/cloudflare-experiments/tree/main/apps/experiments/ai-website-summary">
    View the complete source code
  </Card>

  <Card title="More Experiments" href="/">
    Explore other Cloudflare experiments
  </Card>
</Cards>