AI Batch Infer

Submit a batch of Workers AI embedding inferences with queueRequest: true, then poll by request_id via the Asynchronous Batch API. Without an AI binding, routes return demo queued/poll responses.

FeaturesCopy Anchor Link

• Queue embedding batches (POST /batch)
• Poll results by request_id (GET /batch/:requestId)
• Configurable model via MODEL var (default @cf/baai/bge-m3)

API ReferenceCopy Anchor Link

POST /batchCopy Anchor Link

Body

{
  "texts": ["first phrase", "second phrase"]
}

Example RequestCopy Anchor Link

curl -X POST "https://your-worker.workers.dev/batch" \
  -H "Content-Type: application/json" \
  -d '{"texts":["Cloudflare Workers","edge compute"]}'

Success ResponseCopy Anchor Link

{
  "status": "queued",
  "model": "@cf/baai/bge-m3",
  "request_id": "000-000-000",
  "mode": "live"
}

Error CodesCopy Anchor Link

• 400 - Invalid body (INVALID_BODY)
• 502 - Batch submit failure (BATCH_ERROR)

GET /batch/:requestIdCopy Anchor Link

Poll status/results for a queued batch.

Example RequestCopy Anchor Link

curl "https://your-worker.workers.dev/batch/000-000-000"

While processing, the API may return queued or running. When complete, the response includes per-item results.

Error CodesCopy Anchor Link

• 400 - Invalid id (INVALID_REQUEST_ID)
• 502 - Poll failure (BATCH_ERROR)

Use CasesCopy Anchor Link

• Offline embedding of large document sets
• Batch summarization / classification without interactive latency
• Avoid capacity errors by queueing inference for eventual completion

LimitationsCopy Anchor Link

• Total batch payload must stay under ~10 MB (Workers AI limit)
• Needs a batch-capable model and account AI access for live mode
• Demo mode returns placeholder embeddings

DeploymentCopy Anchor Link

curl -X POST "https://your-worker.workers.dev/batch" \
  -H "Content-Type: application/json" \
  -d '{"texts":["hello","world"]}'

Local DevelopmentCopy Anchor Link

cd apps/experiments/ai-batch-infer
npm install
npm run dev

Workers AI batch typically needs remote AI / account access.

ConfigurationCopy Anchor Link

• Workers AI binding AI
• Var MODEL — batch embedding model id

Cloudflare Features UsedCopy Anchor Link

• Workers - Edge compute runtime
• Workers AI - Inference at the edge
• Asynchronous Batch API - queueRequest + poll by request_id