LlmCall.Dialect.OpenAI.Responses.Connection (llm_call v0.1.0)

Copy Markdown View Source

A WebSocket connection for OpenAI Responses (wss://…/v1/responses).

It owns one WebSocket connection and multiplexes response.create calls over it. Each call names a stream_id lane (generated when omitted); events are decoded per lane by LlmCall.Dialect.OpenAI.Responses and delivered to the %Stream{} returned by create/3.

previous_response_id, store and include belong in the request's provider_options, exactly as on HTTP. steer/3 sends response.steer for the lane's active response id; cancel/2 sends response.cancel and ends the lane. Neither carries stream_id: the provider routes them by response id.

Reconnect

A Responses WebSocket may live for at most 60 minutes. The connection reconnects before that (:max_age_ms, default 55 minutes) and on an unexpected close, with exponential backoff (:retry_delay_ms 5s, :max_retry_delay_ms 60s) and a bounded number of attempts (:max_reconnect_attempts, default 5; :infinity to retry forever).

A failed reconnect does not stop the connection: it becomes :disconnected, create/3 and steer/3 return a :transport error, and the retry loop keeps running until the budget is exhausted. reconnect/1 attempts immediately and resets the budget. Any close triggers the same path — the close code is not part of the decision, so a server that closes because a credential expired is retried until the budget runs out.

With reconnect: false the process instead stops at :max_age_ms, exiting with {:shutdown, :max_age}. A supervisor treats that as a normal stop, but a caller that links it without trapping exits is taken down with it.

An in-flight lane cannot be resumed: its subscriber receives a :transport error. Continuing a conversation across a reconnect is the caller's decision, using previous_response_id and the input it wants to resend.

start_link/1 accepts headers: for the WebSocket handshake; they are merged after the client headers and reused on reconnect.

Delivery

A lane's events are send/2-ed to the process that called create/3, which reduces the returned stream. There is no backpressure: a subscriber that stops reducing while its lane keeps producing will grow its own mailbox. Reduce each lane, or cancel/2 it.

A lane ends when the response finishes, when its subscriber halts the stream (cancel is sent), when the subscriber exits (monitored), or on a connection failure. An unknown or duplicate stream_id is a :validation error, not a crash: a caller mistake never takes down the other lanes.

One lane at a time without an echo

Lanes are routed by the stream_id the server echoes on every event, which the real endpoint does for named lanes. A server that does not echo it can only be attributed while one lane is in flight, so once an event has proven the echo is missing, a second concurrent create/3 is a :validation error instead of a create that would later tear down the lane already running. Back-to-back creates stay allowed while no event has arrived yet: that is how the documented endpoint is used for parallel conversations, and the echo is not known until then.

Summary

Functions

Cancels the lane's active response and ends it.

Returns a specification to start this module under a supervisor.

Closes the WebSocket and stops the process.

Starts one response on stream_id (generated when omitted) and returns its event stream. stream_id must be unique among active lanes.

Reconnects now. In-flight lanes receive a :transport error; idle lanes are unaffected because there are none.

The provider response id for stream_id, if the lane has produced one.

Queues input for the active response on stream_id with response.steer.

Types

t()

@type t() :: GenServer.server()

Functions

cancel(connection, stream_id)

@spec cancel(t(), String.t()) :: :ok | {:error, LlmCall.Error.t()}

Cancels the lane's active response and ends it.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

close(connection)

@spec close(t()) :: :ok

Closes the WebSocket and stops the process.

create(connection, request, opts \\ [])

Starts one response on stream_id (generated when omitted) and returns its event stream. stream_id must be unique among active lanes.

The stream must be reduced by the caller of this function: the connection sends that process the events. Reducing it elsewhere raises ArgumentError.

reconnect(connection)

@spec reconnect(t()) :: :ok | {:error, LlmCall.Error.t()}

Reconnects now. In-flight lanes receive a :transport error; idle lanes are unaffected because there are none.

response_id(connection, stream_id)

@spec response_id(t(), String.t()) :: {:ok, String.t()} | :error

The provider response id for stream_id, if the lane has produced one.

response.created carries it; use it as previous_response_id in a later request. Active lanes answer immediately; completed lanes keep the last 32 ids.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

steer(connection, stream_id, input)

@spec steer(t(), String.t(), term()) :: :ok | {:error, LlmCall.Error.t()}

Queues input for the active response on stream_id with response.steer.

input is wire-shaped (a string or a list of Responses input items); the connection does not reinterpret it. The event carries the lane's previous_response_id, which is what the provider routes a steer by; a lane that has not received response.created yet has no id and is a :validation error.