Streaming responses across the three route shapes

Same flag, three event formats

Streaming is requested the same way everywhere: set stream: true in the request body. What comes back differs by route, because each route speaks its own event format and the proxy does not flatten them into a common one.

On /v1/messages you get the Anthropic event sequence — message_start, content_block_delta, message_delta, message_stop. On /v1/chat/completions you get chat.completion.chunk frames terminated by data: [DONE]. On /v1/responses you get the Responses event stream. Each is what the matching SDK already knows how to iterate.

The response body is passed through rather than buffered, so time-to-first-token is not traded away for the proxy's own bookkeeping. Your request body, by contrast, is read in full before forwarding — it has to be, to determine the model and price the request.

# Anthropic events on /v1/messages
curl -N https://token-share.app/v1/messages \
  -H "x-api-key: $TOKEN_SHARE_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 512,
    "stream": true,
    "messages": [{"role": "user", "content": "Count to five slowly."}]
  }'

# chat.completion.chunk frames, terminated by [DONE]
curl -N https://token-share.app/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN_SHARE_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "grok-4.6",
    "stream": true,
    "messages": [{"role": "user", "content": "Count to five slowly."}]
  }'

# Responses event stream
curl -N https://token-share.app/v1/responses \
  -H "Authorization: Bearer $TOKEN_SHARE_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "gpt-5.6-terra",
    "stream": true,
    "input": "Count to five slowly."
  }'

What happens before the first byte

Once a streaming response is returned, its status line is committed and cannot be changed. So the proxy waits for the first meaningful chunk from the upstream before committing. An upstream that opens 200 text/event-stream and then closes without emitting anything is answered 502 instead of reaching you as an unparseable success you cannot retry.

A newline heartbeat does not count as that first chunk — it is retained so a successful stream is byte-for-byte intact, but it is not treated as evidence that an answer has begun. Only a chunk carrying a non-whitespace byte commits the response.

That single-chunk wait is the whole buffering cost. Nothing further is held back, so the rest of the body flows as fast as the upstream produces it.

Aborting, and what a mid-stream failure looks like

Usage accounting reads the stream on its own branch, split from the one your client consumes. That branch keeps draining after your client disconnects, so aborting just before the final usage event does not hand you the answer for free — the tokens already produced are still counted.

When a provider resets mid-stream, whatever was read before the reset is retained, so a positive usage report that arrived before the failure is still billable. Where no usage was reported at all, the request is recorded as unbilled rather than charged at a guess.

A failure after the status line is committed can only be reported inside the stream, because the 200 is already on the wire. On a stable-* lane a pre-commit failure is emitted as an error event in your route's own format — an Anthropic error event, an OpenAI error frame followed by [DONE], or a response.failed event — so a client mid-iteration ends cleanly instead of hanging on a stream that has already failed.

Common questions