# Newsletter Digest

URL: /docs/experiments/newsletter-digest
> Forward newsletters to Email Routing and get one Workers AI-summarized daily digest - email + AI + cron pipeline
Tags: email, ai, workers-ai, stateful, digest, cron, summarization
Bindings: AI, D1, CRON, EMAIL
Markdown: /llms.mdx/experiments/newsletter-digest/content.md



Forward newsletters to an **Email Routing** address and get one digest email per day instead. An `email()` handler parses each message with `postal-mime`, summarizes it with **Workers AI**, and stores it in **D1**; a daily **Cron Trigger** sends one digest grouped by sender via the `send_email` binding.

## Features [#features]

* Inbound `email()` handler with `postal-mime` parsing; messages over 1 MB are rejected
* Optional `ALLOWED_SENDERS` allowlist (addresses or domains), checked against both envelope sender and `From` header
* HTML-only newsletters are stripped to plain text before summarizing
* 2–3 bullet summaries from `@cf/meta/llama-3.1-8b-instruct-fast`, falling back to the first 300 characters if AI fails
* First non-unsubscribe link kept as a "Read more" link
* Daily digest (text + HTML) grouped by sender; items are marked digested only after the send succeeds

## API Reference [#api-reference]

### GET /items [#get-items]

List the latest 100 items, newest first.

**`pending`** `"true" | "false"` (optional)

`true` returns only items not yet included in a digest.

```bash
curl "https://your-worker.workers.dev/items?pending=true"
```

```json
{
  "count": 1,
  "items": [
    {
      "id": 7,
      "from": "news@example.com",
      "fromName": "Example News",
      "subject": "Issue #42",
      "summary": "- Big launch this week\n- Pricing changes in November",
      "link": "https://example.com/42",
      "receivedAt": "2026-10-01T05:12:00.000Z",
      "digestedAt": null
    }
  ]
}
```

### POST /digest/preview [#post-digestpreview]

Build the digest from pending items (up to 100) without sending or marking anything.

```bash
curl -X POST "https://your-worker.workers.dev/digest/preview"
```

```json
{
  "subject": "Newsletter digest 2026-10-01: 1 item",
  "text": "Newsletter digest 2026-10-01: 1 item\n\n== Example News <news@example.com> ==\n\nIssue #42\n  - Big launch this week\n  - Pricing changes in November\n  https://example.com/42",
  "html": "<!doctype html><html>...</html>",
  "itemCount": 1,
  "senderCount": 1
}
```

### POST /digest/send [#post-digestsend]

Send the digest now to `DIGEST_TO` and mark the included items digested. Requires `Authorization: Bearer <ADMIN_TOKEN>`.

```bash
curl -X POST "https://your-worker.workers.dev/digest/send" \
  -H "Authorization: Bearer $ADMIN_TOKEN"
```

```json
{ "sent": true, "itemCount": 1, "messageId": "..." }
```

When nothing is pending, the response is `{ "sent": false, "itemCount": 0 }` and no email is sent.

#### Error Codes [#error-codes]

* `400` - `INVALID_QUERY` (bad `pending` value), `MISSING_CONFIG` (`DIGEST_TO` not set)
* `401` - `UNAUTHORIZED`
* `502` - `SEND_ERROR` (Email Sending rejected the digest; items stay pending)
* `500` - `INTERNAL_ERROR`

## Architecture [#architecture]

```text
newsletter → Email Routing (digest@yourdomain.com) → email() handler
  ├─ > 1 MB → setReject
  ├─ sender not allowed → ignored
  └─ postal-mime → text / html→text → Workers AI summary → D1 items (digested_at NULL)

cron 0 7 * * * → pending items → group by sender → send_email → digested_at = now
```

## Use Cases [#use-cases]

* Cut a dozen daily newsletters down to one email
* Learn the inbound `email()` handler, Workers AI, D1, Cron, and Email Sending together
* Starting point for any "collect mail, process later" pipeline (receipts, alerts, reports)

## Limitations [#limitations]

* `GET /items` and `POST /digest/preview` are public; put them behind [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/policies/access/) or add the admin check if summaries are sensitive
* HTML-to-text is regex-based and rough on complex layouts; only the first 6000 characters are sent to the model
* Link extraction picks the first non-unsubscribe link, which may be a header or sponsor link
* One digest holds at most 100 items; the rest go in the next run
* Messages are processed one at a time in the `email()` handler, so the AI call adds latency to delivery

## Use in your project [#use-in-your-project]

<UseInYourProject
  experiment="newsletter-digest"
  dependencies="[&#x22;postal-mime&#x22;]"
  bindings="[&#x22;AI&#x22;]"
  platform="[&#x22;Email Workers (`email()` handler)&#x22;, &#x22;Workers AI&#x22;]"
  files="[
  &#x22;src/lib/text.ts&#x22;,
  &#x22;src/lib/summarize.ts&#x22;,
  &#x22;src/constants/defaults.ts&#x22;,
  &#x22;examples/usage.ts&#x22;,
]"
  sources="[{&#x22;path&#x22;:&#x22;src/lib/text.ts&#x22;,&#x22;title&#x22;:&#x22;text.ts&#x22;,&#x22;lang&#x22;:&#x22;ts&#x22;,&#x22;code&#x22;:&#x22;const NAMED_ENTITIES: Record<string, string> = {\n  amp: \&#x22;&\&#x22;,\n  lt: \&#x22;<\&#x22;,\n  gt: \&#x22;>\&#x22;,\n  quot: '\&#x22;',\n  apos: \&#x22;'\&#x22;,\n  nbsp: \&#x22; \&#x22;,\n  mdash: \&#x22;—\&#x22;,\n  ndash: \&#x22;–\&#x22;,\n  hellip: \&#x22;…\&#x22;,\n  rsquo: \&#x22;’\&#x22;,\n  lsquo: \&#x22;‘\&#x22;,\n  rdquo: \&#x22;”\&#x22;,\n  ldquo: \&#x22;“\&#x22;,\n};\n\nconst SKIP_LINK = /unsubscribe|preferences|manage[-_ ]?subscription|view[-_ ]?in[-_ ]?browser/i;\n\nfunction decodeEntities(text: string): string {\n  return text.replace(/&(#x[0-9a-f]+|#\\d+|[a-z]+);/gi, (match, entity: string) => {\n    if (entity[0] === \&#x22;#\&#x22;) {\n      const code =\n        entity[1] === \&#x22;x\&#x22; || entity[1] === \&#x22;X\&#x22;\n          ? Number.parseInt(entity.slice(2), 16)\n          : Number.parseInt(entity.slice(1), 10);\n      return Number.isFinite(code) && code > 0 && code <= 0x10ffff\n        ? String.fromCodePoint(code)\n        : \&#x22;\&#x22;;\n    }\n    return NAMED_ENTITIES[entity.toLowerCase()] ?? match;\n  });\n}\n\n/** Deliberately simple: good enough to feed an LLM, not a faithful renderer. */\nexport function htmlToText(html: string): string {\n  const text = html\n    .replace(/<!--[\\s\\S]*?-->/g, \&#x22;\&#x22;)\n    .replace(/<(head|style|script|noscript|title)\\b[^>]*>[\\s\\S]*?<\\/\\1>/gi, \&#x22;\&#x22;)\n    .replace(/<br\\s*\\/?>/gi, \&#x22;\\n\&#x22;)\n    .replace(/<li\\b[^>]*>/gi, \&#x22;\\n- \&#x22;)\n    .replace(/<\\/(p|div|h[1-6]|ul|ol|tr|table|blockquote|section|article)>/gi, \&#x22;\\n\&#x22;)\n    .replace(/<[^>]+>/g, \&#x22;\&#x22;);\n\n  return decodeEntities(text)\n    .replace(/[\\u200b-\\u200d\\ufeff\\u034f]/g, \&#x22;\&#x22;)\n    .replace(/[ \\t\\f\\v\\u00a0]+/g, \&#x22; \&#x22;)\n    .replace(/ *\\n */g, \&#x22;\\n\&#x22;)\n    .replace(/\\n{3,}/g, \&#x22;\\n\\n\&#x22;)\n    .trim();\n}\n\nexport function extractFirstLink(html: string | undefined, text: string): string | null {\n  const candidates: string[] = [];\n  if (html) {\n    for (const match of html.matchAll(/<a\\b[^>]*\\bhref\\s*=\\s*[\&#x22;']([^\&#x22;']+)[\&#x22;']/gi)) {\n      candidates.push(decodeEntities(match[1]));\n    }\n  }\n  for (const match of text.matchAll(/https?:\\/\\/[^\\s<>()\&#x22;']+/g)) {\n    candidates.push(match[0]);\n  }\n  return candidates.find((url) => /^https?:\\/\\//i.test(url) && !SKIP_LINK.test(url)) ?? null;\n}&#x22;},{&#x22;path&#x22;:&#x22;src/lib/summarize.ts&#x22;,&#x22;title&#x22;:&#x22;summarize.ts&#x22;,&#x22;lang&#x22;:&#x22;ts&#x22;,&#x22;code&#x22;:&#x22;import {\n  AI_INPUT_CHARS,\n  AI_MAX_TOKENS,\n  AI_MODEL,\n  FALLBACK_SUMMARY_CHARS,\n} from \&#x22;../constants/defaults\&#x22;;\nimport type { AiBinding } from \&#x22;../types/env\&#x22;;\n\nconst SYSTEM_PROMPT =\n  \&#x22;You summarize newsletters. Reply with 2 or 3 short bullet points, each starting with '- ', covering the most important points. No preamble, no closing remarks, no links.\&#x22;;\n\nexport function fallbackSummary(text: string): string {\n  const flat = text.replace(/\\s+/g, \&#x22; \&#x22;).trim();\n  if (flat.length <= FALLBACK_SUMMARY_CHARS) return flat;\n  return `${flat.slice(0, FALLBACK_SUMMARY_CHARS).trimEnd()}…`;\n}\n\nfunction normalizeBullets(raw: string): string | null {\n  const bullets = raw\n    .split(\&#x22;\\n\&#x22;)\n    .map((line) => line.trim().replace(/^(?:[-*•]|\\d+[.)])\\s*/, \&#x22;\&#x22;))\n    .filter((line) => line && !/^here (is|are)\\b/i.test(line))\n    .slice(0, 3);\n  return bullets.length ? bullets.map((b) => `- ${b}`).join(\&#x22;\\n\&#x22;) : null;\n}\n\nexport async function summarizeNewsletter(\n  ai: AiBinding,\n  subject: string,\n  text: string\n): Promise<{ summary: string; usedAi: boolean }> {\n  if (!text.trim()) return { summary: subject, usedAi: false };\n  try {\n    const out = await ai.run(AI_MODEL, {\n      messages: [\n        { role: \&#x22;system\&#x22;, content: SYSTEM_PROMPT },\n        { role: \&#x22;user\&#x22;, content: `Subject: ${subject}\\n\\n${text.slice(0, AI_INPUT_CHARS)}` },\n      ],\n      max_tokens: AI_MAX_TOKENS,\n    });\n    const response = (out as { response?: unknown } | null)?.response;\n    const summary = typeof response === \&#x22;string\&#x22; ? normalizeBullets(response) : null;\n    if (summary) return { summary, usedAi: true };\n  } catch (error) {\n    console.error(\&#x22;Workers AI summarization failed\&#x22;, error);\n  }\n  return { summary: fallbackSummary(text), usedAi: false };\n}&#x22;},{&#x22;path&#x22;:&#x22;src/constants/defaults.ts&#x22;,&#x22;title&#x22;:&#x22;defaults.ts&#x22;,&#x22;lang&#x22;:&#x22;ts&#x22;,&#x22;code&#x22;:&#x22;export const AI_MODEL = \&#x22;@cf/meta/llama-3.1-8b-instruct-fast\&#x22;;\nexport const AI_MAX_TOKENS = 200;\nexport const AI_INPUT_CHARS = 6_000;\nexport const FALLBACK_SUMMARY_CHARS = 300;\n\nexport const MAX_EMAIL_BYTES = 1024 * 1024;\nexport const MAX_DIGEST_ITEMS = 100;\nexport const ITEMS_PAGE_SIZE = 100;\n\nexport const DEFAULT_DIGEST_FROM = \&#x22;digest@example.com\&#x22;;&#x22;},{&#x22;path&#x22;:&#x22;examples/usage.ts&#x22;,&#x22;title&#x22;:&#x22;usage.ts&#x22;,&#x22;lang&#x22;:&#x22;ts&#x22;,&#x22;code&#x22;:&#x22;/**\n * Minimal paste example for docs (\&#x22;Use in your project\&#x22;).\n * Not part of the Worker runtime — copy into your own project.\n *\n * Summarizes each inbound email with Workers AI and logs the result.\n */\nimport PostalMime from \&#x22;postal-mime\&#x22;;\nimport { summarizeNewsletter } from \&#x22;../src/lib/summarize\&#x22;;\nimport { extractFirstLink, htmlToText } from \&#x22;../src/lib/text\&#x22;;\nimport type { AiBinding } from \&#x22;../src/types/env\&#x22;;\n\ninterface Env {\n  AI: AiBinding;\n}\n\nexport default {\n  async email(message: ForwardableEmailMessage, env: Env): Promise<void> {\n    if (message.rawSize > 1024 * 1024) {\n      message.setReject(\&#x22;Message exceeds 1 MB limit\&#x22;);\n      return;\n    }\n    const parsed = await PostalMime.parse(await new Response(message.raw).arrayBuffer());\n    const subject = parsed.subject ?? \&#x22;(no subject)\&#x22;;\n    const text = parsed.text?.trim() || htmlToText(parsed.html ?? \&#x22;\&#x22;);\n    const { summary } = await summarizeNewsletter(env.AI, subject, text);\n    console.log({ subject, summary, link: extractFirstLink(parsed.html, text) });\n  },\n};&#x22;}]"
/>

## Deployment [#deployment]

<Steps>
  <Step>
    ### Click the deploy button [#click-the-deploy-button]

    [![Deploy to Cloudflare Workers](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/shrinathsnayak/cloudflare-experiments/tree/main/apps/experiments/newsletter-digest)
  </Step>

  <Step>
    ### Apply migrations and set the admin token [#apply-migrations-and-set-the-admin-token]

    ```bash
    npx wrangler d1 migrations apply newsletter-digest-db --remote
    npx wrangler secret put ADMIN_TOKEN
    ```
  </Step>

  <Step>
    ### Route inbound mail to the Worker [#route-inbound-mail-to-the-worker]

    In the dashboard, enable **Email Routing** on your zone and create a custom address (e.g. `digest@yourdomain.com`) with the action **Send to a Worker** → `newsletter-digest`. Subscribe or forward newsletters to that address.
  </Step>

  <Step>
    ### Configure the digest sender [#configure-the-digest-sender]

    ```bash
    npx wrangler email sending enable yourdomain.com
    ```

    Set `DIGEST_FROM` (on the onboarded domain), `DIGEST_TO`, and optionally `ALLOWED_SENDERS` in `wrangler.json`, then redeploy.
  </Step>

  <Step>
    ### Test your deployment [#test-your-deployment]

    ```bash
    curl "https://your-worker.workers.dev/items?pending=true"
    curl -X POST "https://your-worker.workers.dev/digest/preview"
    ```
  </Step>
</Steps>

## Local Development [#local-development]

```bash
cd apps/experiments/newsletter-digest
npm install
npx wrangler d1 migrations apply newsletter-digest-db --local
npx wrangler dev --test-scheduled
```

Simulate an inbound email, then preview and trigger the cron:

```bash
curl -X POST "http://localhost:8787/cdn-cgi/handler/email?from=news@example.com&to=digest@example.com" \
  --data-binary $'From: Example News <news@example.com>\r\nTo: digest@example.com\r\nSubject: Issue 1\r\nContent-Type: text/plain\r\n\r\nBig news this week. https://example.com/1'

curl -X POST "http://localhost:8787/digest/preview"
curl "http://localhost:8787/__scheduled?cron=0+7+*+*+*"
```

## Configuration [#configuration]

`wrangler.json` declares:

* **Workers AI** `AI`
* **D1** `DB` (`migrations/0000_init.sql` creates `items`)
* **`send_email` binding** `EMAIL`
* **Cron** `0 7 * * *`
* **Vars** `DIGEST_FROM`, `DIGEST_TO`, `ALLOWED_SENDERS` (comma-separated addresses or domains; empty accepts all)

Secret: `ADMIN_TOKEN`. Dependency: `postal-mime`.

## Cloudflare Features Used [#cloudflare-features-used]

* **[Email Routing / Email Workers](https://developers.cloudflare.com/email-routing/email-workers/)** - Inbound `email()` handler
* **[Workers AI](https://developers.cloudflare.com/workers-ai/)** - Newsletter summaries
* **[D1](https://developers.cloudflare.com/d1/)** - Item storage and digest state
* **[Cron Triggers](https://developers.cloudflare.com/workers/configuration/cron-triggers/)** - Daily digest
* **[Email Sending](https://developers.cloudflare.com/email-service/api/send-emails/workers-api/)** - Digest delivery via `send_email`