Rate limits & quotas

Token buckets per API key and per organization, budgets that warn or block before dispatch, and the headers that report both on every response.

#Rate limits

The gateway keeps a token bucket per API key and per organization, each in two dimensions: requests per minute and estimated tokens per minute (the prompt plus max_output_tokens, debited at admission). All four buckets are checked atomically before routing and before the wallet hold, so a rejected request costs nothing and writes no request row; an idempotent replay never touches them. Buckets refill continuously at limit / 60 per second.

A gateway 429 is type: "rate_limit", code: "rate_limit" with param "requests" or "tokens", retriable: true and Retry-After in seconds; the X-RateLimit-* headers describe the bucket that rejected it. A provider's own 429 is code: "provider_rate_limit" and falls back to the next planned attempt inside your constraints.

#Headers

X-Request-Id: req_01J9F3KJ2MQ8ZV5T7XB0N4A6WD      # every response, success or error
X-RateLimit-Limit: 600                            # binding bucket (least headroom): capacity
X-RateLimit-Remaining: 597                        # tokens left in it after this request
X-RateLimit-Reset: 1                              # seconds until it is full again
X-RateLimit-Scope: key                            # key | org
Retry-After: 2                                    # 429 only, seconds until the request would fit
Yelno-Budget-Warning: 8250                        # most-consumed budget, basis points, once ≥ warn_at_bps
Idempotency-Replayed: true                        # stored response for a reused Idempotency-Key
Content-Type: text/event-stream                   # stream: true
Cache-Control: no-cache
X-Accel-Buffering: no

#Budgets

Budgets (/api/v1/budgets, budget:write) cap spend per organization, project or API key over a day, a calendar month or a session, with action: "warn" or "block". Before dispatch the gateway evaluates every enabled budget in scope. A blocking budget whose spend plus the request's estimate exceeds its amount answers 402 type: "insufficient_funds", code: "budget_exceeded", param: "budget" — the message names the budget and the figures; nothing is sent and nothing is charged. Below the limit, once the most-consumed budget reaches its warn_at_bps, every response carries Yelno-Budget-Warning with the projected consumption in basis points (8250 = 82.5%). A warn budget never blocks. Spend lags real usage by at most one worker tick (60 s); the wallet, not the budget, is the money authority.

{
  "error": {
    "type": "insufficient_funds",
    "code": "budget_exceeded",
    "param": "budget",
    "message": "budget bud_… (organization, monthly) is exhausted: 100.00 USD of 100 USD spent, 0.42 USD requested",
    "request_id": "req_…",
    "retriable": false
  }
}

#Payload limits

The gateway sets no body limit of its own; the provider's limits apply, and a provider rejection is returned as 400 invalid_request with code: "provider_http_400" and the provider's message. Each model's context_window and max_output_tokens are listed by GET /v1/models.