Rate Limits & Errors

Rate limits

Limits are enforced per API key over a 60-second sliding window — not per calendar minute, so a burst does not reset at the top of the minute.

Limit	Default per key
Requests per minute	60
Tokens per minute	100,000

Two windows are checked independently:

Requests per minute — counted when the request arrives.
Tokens per minute — a request’s cost is unknown until it finishes, so the window is checked before the call and credited afterwards with the total from usage. A few large completions can therefore throttle the following minute.

Need different limits? Ask an administrator — they are per-key columns, adjustable without new keys.

When you are limited

429 with a Retry-After header, in seconds, derived from when the oldest request in the window expires:


HTTP/1.1 429 Too Many Requests
Retry-After: 42
Content-Type: application/json


{
  "error": {
    "message": "Rate limit exceeded: 60 requests per minute.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "param": null
  }
}

The message states which limit you hit — requests or tokens.

X-RateLimit-Remaining-style headers are not sent yet. Use Retry-After on 429, and track your own consumption from the usage field in each response.

Error format

Every error — including the gateway’s own auth and rate-limit rejections — uses the OpenAI envelope, so SDK error handling works unchanged:


{
  "error": {
    "message": "Human-readable description.",
    "type": "invalid_request_error",
    "code": "invalid_api_key",
    "param": null
  }
}

code is null for request-validation failures; param names the offending field when the gateway can identify it.

Every error code

Status	`type`	`code`	Cause
400	`invalid_request_error`	`null`	Validation failed; `param` names the field
401	`invalid_request_error`	`missing_api_key`	No `Authorization: Bearer` header
401	`invalid_request_error`	`invalid_api_key`	Malformed key, or no such key
403	`invalid_request_error`	`key_revoked`	The key was revoked
404	`invalid_request_error`	`model_not_found`	Unknown or disabled model; `param` is `model`
404	`invalid_request_error`	`unknown_url`	No such endpoint
413	`invalid_request_error`	`null`	Request body over 2 MB
429	`rate_limit_error`	`rate_limit_exceeded`	Requests/min or tokens/min window exhausted
500	`api_error`	`internal_error`	Bug on our side
502	`api_error`	`upstream_error`	The model provider failed or was unreachable

Examples

Validation — note param:


{
  "error": {
    "message": "Too big: expected number to be <=2 (at `temperature`)",
    "type": "invalid_request_error",
    "code": null,
    "param": "temperature"
  }
}

Unknown model:


{
  "error": {
    "message": "The model `gpt-4o` does not exist or you do not have access to it.",
    "type": "invalid_request_error",
    "code": "model_not_found",
    "param": "model"
  }
}

Upstream failure — the provider’s own message is passed through when it sends one:


{
  "error": {
    "message": "model not found",
    "type": "api_error",
    "code": "upstream_error",
    "param": null
  }
}

Retry guidance

Status	Retry?
400, 404, 413	No — fix the request
401	No — fix the key
403	No — the key is revoked; create a new one
429	Yes — wait `Retry-After` seconds
500	Yes — a couple of attempts with backoff
502	Yes — the provider may be briefly unavailable

The OpenAI SDKs already do this. Rolling your own:


import time
import openai
 
def with_retries(call, attempts=4):
    for attempt in range(attempts):
        try:
            return call()
        except openai.RateLimitError as error:
            wait = float(error.response.headers.get("retry-after", 2 ** attempt))
            time.sleep(wait)
        except (openai.APIStatusError, openai.APIConnectionError) as error:
            status = getattr(error, "status_code", 500)
            if status < 500:
                raise            # 4xx will not fix itself
            time.sleep(2 ** attempt)
    raise RuntimeError("out of retries")

Retry 429 and 5xx only, respect Retry-After, and add jitter if many workers share one key — otherwise they all retry in lockstep and hit the limit together.

Streaming errors

Errors before the first chunk arrive as JSON with a status code. Once the stream has started the status is already 200, so a later failure just ends the stream: a stream that stops without a finish_reason is incomplete. See Streaming.