Responses

The OpenAI Responses dialect, the default of the newest OpenAI SDKs. This gateway implements the stateless part: you send the input, you get one answer, and nothing is kept. Anything that needs the server to remember or to act returns 501.

POST/v1/responses#

• Auth and limit: like chat completions: your API key, and the shared /v1 request limit.
• Body: JSON, with Content-Type: application/json.

curl -s "$SHVIA_URL/v1/responses" \
  -H "Authorization: Bearer $SHVIA_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "'"$SHVIA_MODEL"'",
    "instructions": "Answer in one sentence.",
    "input": "What is a gateway?"
  }'

Request#

An array input holds messages with a role of user, assistant, system or developer (the last two both become system), and a content that is a string or a list of input_text, output_text or text blocks. The input must contain at least one user message, or the call returns 400.

What returns 501#

These are refused, with the body shown below, before the model is chosen:

{
  "error": {
    "message": "'tools' is not supported by this gateway yet (stateless slice — see docs).",
    "type": "api_error",
    "param": null,
    "code": "not_implemented"
  }
}

The quoted name in message is the first field that was refused.

Response#

{
  "id": "resp_2c058e36a9f2418398cebdcd",
  "object": "response",
  "created_at": 1767225600,
  "status": "completed",
  "model": "claude-sonnet-4-6@anthropic",
  "output": [
    {
      "type": "message",
      "id": "msg_2c058e36a9f2418398cebdcd",
      "status": "completed",
      "role": "assistant",
      "content": [{ "type": "output_text", "text": "A gateway is a single entry point in front of several services.", "annotations": [] }]
    }
  ],
  "usage": {
    "input_tokens": 21,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 14,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 35
  },
  "store": false,
  "error": null,
  "incomplete_details": null,
  "instructions": "Answer in one sentence.",
  "metadata": {},
  "parallel_tool_calls": true,
  "tool_choice": "auto",
  "tools": []
}

• A completed response holds exactly one message item with one output_text part. The in-progress and failed objects of a stream have an empty output.
• parallel_tool_calls, tool_choice and tools are constants.
• instructions is null when you sent none, and metadata is {}.

Streaming#

With "stream": true each event is event: <type> and data: {…}, and every payload carries a sequence_number that starts at 0. The events come in this order:

1. response.created and response.in_progress, with the response object in progress.
2. When the model produces text: response.output_item.added, response.content_part.added, then one response.output_text.delta per fragment.
3. response.output_text.done, response.content_part.done and response.output_item.done, which carry the full text.
4. One last event: response.completed, response.incomplete or response.failed.

event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_…","output_index":0,"content_index":0,"delta":"A gateway","sequence_number":4}

event: response.output_text.done
data: {"type":"response.output_text.done","item_id":"msg_…","output_index":0,"content_index":0,"text":"A gateway is a single entry point.","sequence_number":7}

When the model stops answering mid-reply, the text part is closed, and the stream ends with response.incomplete and incomplete_details.reason set to read_timeout. When the model returns nothing, or the servers cannot be reached, it ends with response.failed, and response.error.code is empty_response or upstream_error.

Errors#

Errors use the OpenAI shape, as in chat completions: 401 invalid_api_key, 403 permission_denied, 404 model_not_found, 400 for a bad body, 501 not_implemented, and 502 or 504 with empty_response, upstream_error or read_timeout. A 504 adds a partial object, the response as far as it got, and the header X-Inference-Incomplete: read_timeout. When every model of the chain fails, a refusal from a model provider keeps the provider's own status and the code upstream_error. A 401 or 403 here can therefore come from the model provider, for example a model server that rejects the gateway's own credentials, and not from your key: check the key with GET /api/v1/auth/verify.

GET/v1/responses/{id}#

Always 404, because nothing is stored:

{
  "error": {
    "message": "Response 'resp_x' was not stored: this gateway is stateless (store is not supported yet).",
    "type": "invalid_request_error",
    "param": null,
    "code": "not_found_error"
  }
}

There is no other route for responses: no delete, no cancel and no input listing.

Data handling#

Like chat completions: nothing is stored as a conversation, the call is metered, and common identifiers in the text are masked before it goes to a model that is not on the operator's own infrastructure. See your data.