LlmCall.Stream (llm_call v0.1.0)

Copy Markdown View Source

A lazy Enumerable over a streaming call.

Built by LlmCall.stream/3, which has already encoded the request and started the codec's stream state, so an invalid request is a tagged error. The HTTP request starts when the stream is reduced.

The struct is an immutable description, not a cursor: reducing it twice issues the request twice. Build it once and pass it to one consumer.

Refer to it as LlmCall.Stream rather than alias-ing it as Stream, which would shadow Elixir's Stream module for the rest of that module.

Errors

A 2xx body's events are yielded as they decode. Everything that fails while the body is being read raises LlmCall.HTTP.Error: a non-2xx status (after the error body is decoded by the codec), a payload the JSON module cannot parse, or a {:error, %LlmCall.Error{}} from decode_stream_event/3. An Enumerable cannot return a tuple at that point.

A codec that breaks the LlmCall.Codec contract is reported the same way, as :invalid_response: an exception raised from decode_stream_event/3 or finish_stream/2, or a return value outside {:ok, events, state}, {:done, state}, and {:error, %LlmCall.Error{}}.

{:halt, acc} from the reducer stops the request and closes the connection; finish_stream/2 is not called for a halted consumer. Suspension ({:suspend, acc}) is not supported and raises.

on_event

LlmCall.stream/3 accepts on_event: fun with fun :: (StreamEvent.t() -> any()). It runs for every event before the reducer sees it, including events flushed by finish_stream/2. The return value is ignored: it observes, it cannot filter or replace events, and it is not called after the reducer halts. If it raises, the exception propagates and the reducer does not see that event.

Summary

Types

on_event()

@type on_event() :: (LlmCall.StreamEvent.t() -> any()) | nil

t()

@type t() :: %LlmCall.Stream{
  client: LlmCall.Client.t(),
  codec: module(),
  http_request: LlmCall.HTTP.Request.t(),
  on_event: on_event(),
  state: LlmCall.StreamState.t()
}