Auto (cost optimized)
Send "model": "auto" and the gateway picks a real model for you, aiming at the cheapest one that
can actually do the job. It is a drop-in substitution — nothing else about the request changes.
curl https://api.devotel.com/v1/chat/completions \
-H "Authorization: Bearer $DEVOTEL_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"reply with OK"}]}'The response tells you which model answered:
{ "model": "deepseek-4-flash", "choices": [...] }What it optimises, and what it will not trade away
The router looks at the shape of your first request — how much context, how many messages, whether there are tools or images, how much code, whether you asked for deep reasoning — and picks a tier. Most short questions do not need an expensive model, and that is where the saving comes from.
Capability is decided before cost. A request with tools only goes to a model that has been
measured to emit well-formed tool_calls; a request with images only goes to one measured to
actually read them. Those flags come from the models table, where they are measured
against the live provider rather than taken from a datasheet. Cheap is worthless if the answer is
wrong.
A tiny max_tokens can still land you on a model that reasons by default — the MID tier, in
particular — so the same max_tokens/reasoning_effort guidance
applies to auto too, regardless of which model it ends up picking.
Your conversation keeps its model
Once a conversation is assigned a model, it keeps it. This is not a detail — it is most of why auto
is cheaper rather than just different:
- The provider caches your prompt prefix, and cache reads cost roughly a tenth of fresh input. Around 92% of our measured input volume is cache reads. Moving a conversation to another model throws that away and re-pays full price for the entire prefix.
- Tool-call history and reasoning context do not necessarily transfer between models mid-loop.
The assignment is keyed on your system prompt plus your first user message, so turn 20 of a conversation resolves to the same model as turn 1. It expires after 24 hours of inactivity.
Escalation, and the one retry
If the assigned model fails — the provider returns a 5xx or a rate limit before any token has been sent — the request is retried once, and the conversation is re-pinned wherever it lands. It never moves back down to a cheaper tier.
Where that one retry goes depends on whether there is anywhere above:
- Up, to the next tier, whenever one exists. This is the normal case.
- Sideways, to an equally-capable model in the same tier, when there is no tier above — the top of the ladder having an outage is the case this exists for. A sideways target must match the original on every capability your request needs, so a tools-and-images request never lands somewhere that cannot serve it.
A rate limit (429) only ever moves up, never sideways: it means “not right now”, and a sibling
model in the same tier has no more quota to offer than the one that just refused.
One retry is the whole budget, deliberately. Chaining “up, then sideways” would mean three upstream attempts on a request that is already slow, and during the outage this was built for each failed attempt cost about a second — tripling load on a provider that was already struggling.
A request that has already started streaming is never re-routed. Once the first token reaches you, the model for that request is fixed: splicing two models’ output into one answer would be worse than the failure.
A provider 400 is not retried. That is a statement about your request, and it would be just as
wrong one tier up — you get the error, with the provider’s own message.
Restricting which models auto may use
auto respects per-key model access like any other model. If your key has a model list, auto must
be on it, and the router will only ever choose from the intersection of the ladder and your list.
key allows: auto, glm-5.2 -> auto routes to glm-5.2 only
key allows: everything (default) -> auto uses the full ladderIt never routes outside your list, and it never silently degrades. If nothing on your list can serve
a request — an image on a key limited to text-only models, say — you get a 400 naming the missing
capability rather than a quietly worse answer:
{
"error": {
"message": "No model available to this key supports image input. Name a model explicitly, or ask for access to one.",
"type": "invalid_request_error",
"param": "model"
}
}A restricted model is never a routing target, even if your key has been granted it. A grant lets you call that model by name; it does not send traffic there on your behalf.
Billing shows the real model
Usage and cost always record the model that actually ran, never auto. On your
usage dashboard a routed request appears under deepseek-4-flash
or kimi-k3 — whichever answered — priced at that model’s rate. auto has no price of its own,
because it never bills anything.
That means switching to auto changes what you are charged, not how you read the charges.
When not to use it
- You need one specific model’s behaviour reproducibly — name it.
- You are benchmarking models against each other.
- You have your own routing logic and want this gateway to be a plain pass-through.
Naming a model explicitly bypasses the router completely; it is not consulted at all.