Streaming

Set "stream": true and the response arrives as server-sent events instead of one JSON body.

Wire format

Each event is a data: line holding one chunk, separated by a blank line, and the stream ends with data: [DONE]:


data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","reasoning_content":"The user asked"}}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" there"}}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":95,"completion_tokens":158,"total_tokens":253}}

data: [DONE]

Three things the gateway guarantees:

Chunks are forwarded byte-for-byte. Nothing is rewritten, buffered or reordered.
The usage chunk always arrives. stream_options.include_usage is set on every upstream stream, so the second-to-last event carries the token counts even if you did not ask for them. That chunk has an empty choices array — read usage, not choices[0].
data: [DONE] always terminates the stream, including when the provider omits it.

Reasoning arrives first

Kimi K3 streams delta.reasoning_content before it streams delta.content. A real request emitted 790 chunks of reasoning before the first character of the answer, so expect a pause before visible output and accumulate the two fields separately:


let content = ''
let reasoning = ''
 
// per chunk:
content += chunk.choices[0]?.delta?.content ?? ''
reasoning += chunk.choices[0]?.delta?.reasoning_content ?? ''

Show a “thinking” indicator while only reasoning_content is arriving — otherwise the UI looks frozen for the first few seconds.

Examples

curl


curl -N https://api.devotel.com/v1/chat/completions \
  -H "Authorization: Bearer $DEVOTEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kimi-k3",
    "messages": [{"role": "user", "content": "Count to five."}],
    "stream": true
  }'

-N disables curl’s buffering so you see events as they arrive.

Python


import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["DEVOTEL_API_KEY"],
    base_url="https://api.devotel.com/v1",
)
 
stream = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
 
reasoning, content = "", ""
 
for chunk in stream:
    # The final usage chunk has no choices.
    if chunk.usage and not chunk.choices:
        print(f"\n\n[{chunk.usage.completion_tokens} output tokens]")
        continue
 
    delta = chunk.choices[0].delta
 
    # reasoning_content is not part of the OpenAI SDK's typed model; read it dynamically.
    piece = getattr(delta, "reasoning_content", None)
    if piece:
        reasoning += piece
        print(".", end="", flush=True)   # thinking indicator
 
    if delta.content:
        content += delta.content
        print(delta.content, end="", flush=True)

JavaScript


import OpenAI from 'openai'
 
const client = new OpenAI({
  apiKey: process.env.DEVOTEL_API_KEY,
  baseURL: 'https://api.devotel.com/v1',
})
 
const stream = await client.chat.completions.create({
  model: 'kimi-k3',
  messages: [{ role: 'user', content: 'Count to five.' }],
  stream: true,
})
 
let reasoning = ''
let content = ''
 
for await (const chunk of stream) {
  // The final usage chunk has no choices.
  if (chunk.usage && chunk.choices.length === 0) {
    console.log(`\n\n[${chunk.usage.completion_tokens} output tokens]`)
    continue
  }
 
  const delta = chunk.choices[0]?.delta as
    { content?: string; reasoning_content?: string } | undefined
 
  if (delta?.reasoning_content) {
    reasoning += delta.reasoning_content
    process.stdout.write('.') // thinking indicator
  }
 
  if (delta?.content) {
    content += delta.content
    process.stdout.write(delta.content)
  }
}

Reading the stream without an SDK

If you parse SSE yourself, buffer by line — a single TCP read can contain half an event:


const response = await fetch('https://api.devotel.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.DEVOTEL_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'kimi-k3',
    messages: [{ role: 'user', content: 'Count to five.' }],
    stream: true,
  }),
})
 
const reader = response.body!.getReader()
const decoder = new TextDecoder()
let buffer = ''
 
for (;;) {
  const { done, value } = await reader.read()
  if (done) break
 
  buffer += decoder.decode(value, { stream: true })
 
  let newline = buffer.indexOf('\n')
  while (newline !== -1) {
    const line = buffer.slice(0, newline).trim()
    buffer = buffer.slice(newline + 1)
    newline = buffer.indexOf('\n')
 
    if (!line.startsWith('data:')) continue
    const data = line.slice(5).trim()
    if (data === '[DONE]') continue
 
    const chunk = JSON.parse(data)
    process.stdout.write(chunk.choices?.[0]?.delta?.content ?? '')
  }
}

Errors mid-stream

Errors that happen before the first chunk come back as a normal JSON error with an HTTP status — see Rate Limits & Errors. Once streaming has started the status is already 200, so a provider failure ends the stream early; treat a stream that stops without finish_reason as incomplete and retry if it matters.

If you disconnect mid-stream, the gateway aborts its upstream request rather than letting the generation run to completion. Devotel meters only the tokens reported before the stream ended.