Fixture-based assertions for codec authors.
These are test support, not a runtime dependency: a third-party codec can call
them from its own suite to check the parts of the behaviour that are easy to
get subtly wrong. Every function returns :ok or raises an
ExUnit.AssertionError naming the failed expectation.
test "my codec conforms" do
Conformance.assert_info(MyCodec)
Conformance.assert_path(MyCodec, request, "/v1/my-endpoint")
endThe assertions take the caller's fixtures, so an out-of-tree codec supplies its own payloads: nothing here assumes a specific provider.
Summary
Functions
Couples capabilities.streaming to the streaming callbacks.
Asserts that capabilities/2 and validate_request/2 agree, in both
directions.
Asserts that a provider error envelope decodes to a canonical %Error{}.
Asserts that a successful payload decodes to a canonical response.
Asserts well-formed info/0 metadata (see LlmCall.Conformance).
Asserts that a malformed payload is a typed error, never a raise and never a fabricated response.
Asserts the encoded request's method, path, body shape, and credential-free headers.
Asserts that a codec rejects a request it cannot represent in full.
Asserts which decoded content parts a codec accepts back in an assistant message.
Drives a codec's stream over the caller's chunks and asserts event shape and order.
Asserts that StreamEvent.raw follows the capture_raw policy.
Asserts that a streaming request actually asks the provider to stream.
Asserts that a stream state is bound to the codec that created it.
Asserts that a streamed tool-call sequence assembles into complete calls.
Functions
@spec assert_capabilities(module(), LlmCall.Request.t(), LlmCall.Config.t()) :: :ok
Couples capabilities.streaming to the streaming callbacks.
streaming: true requires all three callbacks; streaming: false requires
none of them. Capabilities are per codec, not per request, so a protocol that
streams only for some models cannot express that here — declare true and
reject the other requests in validate_request/2.
@spec assert_capability_consistency(module(), LlmCall.Request.t(), LlmCall.Config.t()) :: :ok
Asserts that capabilities/2 and validate_request/2 agree, in both
directions.
A declared capability must accept a minimal representative: tools,
tool_choice, structured_output, modalities, image_input, file_input,
reasoning.supported, every declared efforts level, budget_tokens, and
summary. An undeclared one must be a validation error. The base request must
otherwise be valid for the codec.
reasoning.adaptive, reasoning.default_effort, parallel_tool_calls,
streaming_tool_calls, prompt_cache, and stateful are declarations only:
the canonical IR has no request field that expresses them, so there is nothing
to check in either direction.
@spec assert_capture_raw(module(), LlmCall.wire_value()) :: :ok
@spec assert_decode_error(module(), LlmCall.wire_value(), keyword()) :: :ok
Asserts that a provider error envelope decodes to a canonical %Error{}.
Pass code: and retryable?: to pin the normalization: without them the
assertion only sees that the result is a provider error, and a codec that
ignores the payload entirely would pass.
@spec assert_decode_response(module(), LlmCall.wire_value()) :: :ok
Asserts that a successful payload decodes to a canonical response.
Checks the IR boundary: canonical parts, a stop reason from the closed union,
canonical usage, and raw as a map or nil. Provider-specific shapes leaking
through content fail here.
@spec assert_info(module()) :: :ok
Asserts well-formed info/0 metadata (see LlmCall.Conformance).
@spec assert_invalid_response(module(), LlmCall.wire_value()) :: :ok
Asserts that a malformed payload is a typed error, never a raise and never a fabricated response.
Call it with a payload the codec cannot decode — %{} for most protocols.
@spec assert_path(module(), LlmCall.Request.t(), String.t(), :post | :get) :: :ok
Asserts the encoded request's method, path, body shape, and credential-free headers.
method defaults to :post. A codec that uses encode: false for a
per-request method passes the one it produces.
@spec assert_rejects(module(), LlmCall.Request.t(), LlmCall.Config.t()) :: :ok
Asserts that a codec rejects a request it cannot represent in full.
The core promise is that an unrepresentable canonical field is a validation error, never a silent drop. This is the assertion a package uses to prove it, and the one that pins a protocol's unsupported fields as a test rather than a convention.
@spec assert_replays(module(), LlmCall.Request.t(), keyword()) :: :ok
Asserts which decoded content parts a codec accepts back in an assistant message.
A response the caller can decode but not replay makes a tool loop impossible, so the accept/reject boundary is part of the contract rather than an implementation detail. Pass the rule per part and the assertion builds the assistant turn itself:
Conformance.assert_replays(codec, request,
text: :ok,
tool_call: :ok,
thinking: :rejected,
thinking_without_signature: :rejected,
raw: :rejected
):ok means encode_request/2 succeeds with that part in an assistant message;
:rejected means it is a :validation error that names the part — never a silent
drop. A part the caller omits is not checked: a protocol that cannot express it at
all has no rule to pin.
The parts are :text, :thinking, :thinking_without_signature, :tool_call, and
:raw. What each codec answers is the table in docs/design.md §8.2.
@spec assert_reserved_options(module(), LlmCall.Request.t(), LlmCall.Config.t(), [ map() ]) :: :ok
@spec assert_stream(module(), LlmCall.Request.t(), [LlmCall.wire_value()]) :: :ok
Drives a codec's stream over the caller's chunks and asserts event shape and order.
Order rules: :message_start at most once and before any delta;
:message_stop/:error at most once and terminal. A {:done, state} result
stops the stream, and finish_stream/2 afterwards must not emit content.
@spec assert_stream_capture_raw(module(), LlmCall.Request.t(), [LlmCall.wire_value()]) :: :ok
Asserts that StreamEvent.raw follows the capture_raw policy.
:none and :response keep no provider event; :all keeps at least one, and
every retained payload is a map. A codec that copies provider chunks into every
event regardless of the policy fails here.
@spec assert_stream_request(module(), LlmCall.Request.t(), keyword()) :: :ok
Asserts that a streaming request actually asks the provider to stream.
This is the encoding-side counterpart to assert_stream/3. A codec can
implement every streaming callback and still never request SSE, in which case
the transport receives a buffered response and the decoders are dead code.
Three things must hold:
stream: trueandstream: falsemust encode to different requests, so the flag reaches the wire at all;- the streaming request must carry a streaming signal:
body["stream"] == true, an SSE query value (alt=sse), astreamquery flag, or a path that differs from the non-streaming one and names the streaming endpoint; - the non-streaming request must not carry that signal.
A signal is a disjunction, so this cannot tell you that a protocol needs all
of its signals (Gemini needs both alt=sse and the streaming path). Pass
require: [query: %{"alt" => "sse"}] or require: [body: %{...}] to pin the
ones your protocol cannot lose.
Pass marker: (1-arity, receives the streaming request) when the protocol
signals streaming some other way, and config: when the codec needs a
non-default LlmCall.Config to encode. The base request must be streamable:
forcing stream: true on a request the codec rejects (Gemini with an image
modality, say) is reported as such, and a codec that declares
streaming: false is named as the cause.
@spec assert_stream_roundtrip(module(), LlmCall.Request.t()) :: :ok
@spec assert_stream_state_ownership(module(), module(), LlmCall.Request.t()) :: :ok
Asserts that a stream state is bound to the codec that created it.
Passing one codec's state to another must be a validation error, so a caller cannot mix two protocols' stream state by accident.
@spec assert_tool_call_assembly( module(), LlmCall.Request.t(), [LlmCall.wire_value()], pos_integer(), keyword() ) :: :ok
Asserts that a streamed tool-call sequence assembles into complete calls.
Runs the events through LlmCall.StreamAssembler, so a codec fails here if
it fragments arguments in a way the assembler cannot key by index, or invents
arguments the provider never sent.
expected_calls pins the count; pass json_decoder: to supply a decoder other
than :json (the assembler never decodes JSON itself).