Cloud AI Proxy

The Cloud AI Proxy experiment exposes Workers AI through a single public endpoint. You pass a model ID and prompt (or chat messages) and receive AI-generated text. Use it to try different models or integrate Workers AI from external tools without managing Wrangler or bindings yourself.

FeaturesCopy Anchor Link

• Call any Workers AI text-generation model via one endpoint
• Support for single prompt or chat-style messages
• Optional max_tokens control
• POST (JSON) and GET (query params) interfaces
• No external API keys; uses your Cloudflare account’s Workers AI

API ReferenceCopy Anchor Link

POST /chatCopy Anchor Link

Send a JSON body with model and prompt or messages.

*At least one of prompt or messages is required.

response string

The AI-generated text.

Example RequestCopy Anchor Link

curl -X POST https://your-worker.workers.dev/chat \
  -H "Content-Type: application/json" \
  -d '{"model":"@cf/meta/llama-3.1-8b-instruct-fast","prompt":"Say hello in one sentence."}'

Example ResponseCopy Anchor Link

{
  "response": "Hello! How can I help you today?"
}

GET /chatCopy Anchor Link

Query params: model (required), prompt (required), optional max_tokens.

Example RequestCopy Anchor Link

curl "https://your-worker.workers.dev/chat?model=@cf/meta/llama-3.1-8b-instruct-fast&prompt=Say%20hello"

Error ResponsesCopy Anchor Link

• 400 Bad Request - INVALID_BODY, MISSING_MODEL, MISSING_PROMPT, or INVALID_MAX_TOKENS
• 502 Bad Gateway - AI_ERROR (model run failed)

Use CasesCopy Anchor Link

• Try Workers AI models from curl, Postman, or external tools without managing bindings
• Prototype chat APIs before adding authentication and rate limiting
• Compare model outputs by swapping the model parameter
• Integrate edge AI into apps via a simple HTTP proxy

LimitationsCopy Anchor Link

• Public endpoint with no authentication or rate limiting by default
• Workers AI is subject to usage limits by plan
• max_tokens is capped at 4096
• Text generation only; no streaming responses or tool calling

Use in your projectCopy Anchor Link

import type { Env } from "../types/env";type RunOptions = {  prompt?: string;  messages?: Array<{ role: string; content: string }>;  max_tokens?: number;};export async function runChat(env: Env, model: string, options: RunOptions): Promise<string> {  const out = await env.AI.run(model, {    prompt: options.prompt,    messages: options.messages,    max_tokens: options.max_tokens,  });  const response =    (out as { response?: string }).response ?? (out as { result?: string }).result ?? "";  return response;}

DeploymentCopy Anchor Link

curl -X POST "https://your-worker.workers.dev/chat" \
  -H "Content-Type: application/json" \
  -d '{"model":"@cf/meta/llama-3.1-8b-instruct-fast","prompt":"Say hello in one sentence."}'

Local DevelopmentCopy Anchor Link

cd apps/experiments/cloud-ai-proxy
npm install
npm run dev

Call the endpoint at http://localhost:8787/chat with POST (JSON body) or GET (query params). Requires a Cloudflare account with Workers AI enabled.

Cloudflare Features UsedCopy Anchor Link

• Workers - Edge runtime
• Workers AI - AI binding for text generation

Next StepsCopy Anchor Link