Blog22 min read
Build a RAG search API with Vectorize and Workers AI
Deep RAG on Cloudflare - Workers AI embeddings, Vectorize indexes, chunking, citations, AI Search vs DIY, batch ingest, and RAG Mini Search.
RAG (retrieval-augmented generation) lets you answer questions from your documents instead of hoping the model memorized them. On Cloudflare, the usual spine is:
- Chunk and embed content with Workers AI
- Store vectors in Vectorize
- Query top-k chunks at request time
- Ask a text model to answer using those chunks
The RAG Mini Search experiment packages that pattern as a deployable reference.
Cloudflare products in this pattern
| Product | Role | Docs |
|---|---|---|
| Workers AI | Embedding + generation models | AI inference |
| Workers AI models | Pick embedders / LLMs | Model catalog |
| Vectorize | Vector index + query | Vector DB |
| Vectorize Workers API | Upsert / query from Workers | Get started |
| AI Search | Managed RAG alternative | Managed product |
| AI Gateway | Observability / cache | Gateway |
| Queues | Async ingest | Background jobs |
| R2 | Store raw source docs | Objects |
| Workers | Query + ingest HTTP APIs | Runtime |
When RAG is worth it
Good fits
- Docs Q&A for a product catalog
- Internal runbooks with citations
- Per-tenant knowledge bases with isolated indexes / namespaces
Skip RAG when
- Keyword search or Text Similarity is enough
- You only need a one-off URL summary (AI Website Summary)
- Documents change every second and you lack an ingest pipeline
Architecture
Ingest: docs → chunk → embed (Workers AI) → Vectorize upsert
Query: question → embed → Vectorize query → prompt + chunks → Workers AI → answerKeep ingestion off the hot path when possible (AI Batch Infer, Queues). The query Worker should stay under ~60 seconds.
Official Vectorize path: Get started / embeddings.
Design choices that matter
Chunking
- Prefer semantic chunks (roughly 200–800 tokens) over whole pages
- Store
sourceUrl/titlemetadata with each vector for citations (metadata filters) - Re-embed on content change; include a content hash
- Generate LLM-friendly text first with Website to llms.txt when sources are HTML
Retrieval
- Start with top-k = 3–8
- Filter by tenant/namespace metadata
- Log empty retrievals - they produce hallucinations if you still call the LLM
- Tune similarity metric to match how you created the index (Vectorize indexes)
Generation
- Instruct the model to say “I don’t know” when chunks are irrelevant
- Return citations alongside the answer
- Cap max tokens for predictable latency
- Optionally front models with AI Gateway
DIY Vectorize vs AI Search
| Approach | Experiment | Product |
|---|---|---|
| Full control over chunking/prompts | RAG Mini Search | Vectorize + Workers AI |
| Managed RAG | AI Search Demo | AI Search |
Use Mini Search when you need full control; use AI Search when operational simplicity wins.
Ingest at scale
- Accept documents via authenticated API
- Enqueue chunk jobs on Queues
- Consumer embeds with Workers AI and upserts to Vectorize
- Store originals in R2 for reprocessing
AI Batch Infer demonstrates async embedding-oriented patterns you can adapt.
Evaluation checklist
- Questions only your docs can answer (expect citations)
- Questions outside the corpus (expect refusal)
- Near-duplicate chunks (dedupe strategy)
- Multi-tenant leakage tests (metadata filters)
Deploy RAG Mini Search
- Create a Vectorize index (create indexes)
- Enable Workers AI binding (Workers AI get started)
- Deploy RAG Mini Search
- Ingest a small doc set
- Ask a question that only your docs can answer
- Verify citations and empty-corpus behavior
- Compare with AI Search Demo if you want managed RAG
You leave with a working RAG API on the edge - enough to validate product UX before investing in larger ingestion pipelines - built on Vectorize and Workers AI as first-class Cloudflare products.
Production checklist
- Create the Vectorize index with the correct dimensions for your embedder.
- Store rich metadata for citations and tenant filters.
- Keep ingest async via Queues.
- Evaluate with questions inside and outside the corpus.
- Compare DIY vs AI Search before building a large pipeline.
- Cap generation tokens and always return citations.
FAQ
Embeddings model changed - what now?
You generally need a new index (or full re-embed). Pin model ids and treat changes as migrations.
How many chunks per query?
Start with top-k 3–8. More chunks raise cost and can dilute the prompt.
Related experiments
These deployable references implement the patterns in this article. Open the docs for API details, then Deploy or clone the source.
- RAG Mini Search - Mini RAG: Vectorize retrieval + Workers AI grounded answers over experiment docs - retrieval-augmented generation reference
- AI Search Demo - Query Cloudflare AI Search (managed RAG) from a Worker - grounded answers with retrieval over your indexed content
- Text Similarity - Compare text similarity with Workers AI embeddings at the edge - cosine similarity for duplicate and related-content detection
- AI Batch Infer - Workers AI async batch embeddings with queueRequest and polling - process large embedding jobs without blocking the request path
- Website to llms.txt - Convert any webpage into llms.txt format for LLM and AI-agent consumption - AI-readable site summary generator
- AI Website Summary - Summarize any webpage with Workers AI on Cloudflare's edge - fetch URL, extract content, return a concise AI summary