A call-oriented SDK for LLM APIs: a canonical IR, dialect codecs, and a one-call client on Finch whose JSON layer you can replace.

defmodule MyApp.OpenAI do
  use LlmCall.Provider,
    base_url: "https://api.openai.com",
    auth: :bearer,
    codecs: [chat: LlmCall.Dialect.OpenAI.Chat]
end

client = MyApp.OpenAI.client(api_key: System.fetch_env!("OPENAI_API_KEY"))

request = %LlmCall.Request{
  model: "gpt-4o",
  messages: [%LlmCall.Message{role: :user, content: [%LlmCall.Content.Text{text: "Hello"}]}]
}

{:ok, response} = LlmCall.complete(client, request)
LlmCall.Response.text(response)

complete/3 validates, encodes, sends, and decodes. stream/3 does the same and returns a lazy Enumerable of events. generate/3 and poll/3 cover dedicated image and video endpoints, including long-running jobs.

Installation

def deps do
  [
    {:llm_call, "~> 0.1.0"}
  ]
end

Requires Elixir 1.18+ on OTP 27+. Runtime dependencies are Finch and mint_web_socket; everything else is dev/test only.

Providers

A provider is a module you write: endpoint, auth shape, and the codecs it serves. There is no built-in catalog — examples/ carries reference definitions.

defmodule MyApp.Groq do
  use LlmCall.Provider,
    base_url: "https://api.groq.com/openai",
    auth: :bearer,
    codecs: [chat: LlmCall.Dialect.OpenAI.Chat]
end

client = MyApp.Groq.client(api_key: key)

codecs: is a named map because one endpoint can serve several request shapes. An endpoint that returns text and images is still one chat codec (modalities: [:text, :image]); a different request shape — Imagen's predict, OpenAI Images — is a second codec selected per call:

{:ok, %LlmCall.Media.Response{assets: assets}} =
  LlmCall.generate(client, media_request, codec: :images)

Auth shapes: :none, :bearer, :x_api_key, {:header, name}, or {module, function}. The key is always passed in; nothing reads the environment.

query: adds URL parameters to every request, for vendors that key on one — MiniMax's GroupId, Azure OpenAI's api-version. A codec's own parameter still wins on the same key, and the Responses WebSocket handshake gets them too. A query already written into base_url is folded into query: when the client is built, so pasting https://…/openai?api-version=2024-02-01 works.

defmodule MyApp.MiniMax do
  use LlmCall.Provider,
    base_url: "https://api.minimax.io",
    auth: :bearer,
    codecs: [video: LlmCall.Media.Dialect.MiniMax.Video]
end

client =
  MyApp.MiniMax.client(
    api_key: System.fetch_env!("MINIMAX_API_KEY"),
    query: %{"GroupId" => System.fetch_env!("MINIMAX_GROUP_ID")}
  )

The declaration only takes literals, so a value read at runtime belongs on client/1.

Streaming

{:ok, stream} = LlmCall.stream(client, %{request | stream: true})

stream
|> Stream.each(&IO.inspect({&1.type, &1.delta}))
|> Stream.run()

The library parses SSE and provider terminators. A failure while the body is being read raises LlmCall.HTTP.Error, whose error/1 returns the canonical %LlmCall.Error{} with the HTTP status; {:halt, acc} from a reducer closes the connection. complete/3 rejects stream: true rather than silently flipping it.

To observe events without owning the reduce, pass on_event::

{:ok, stream} =
  LlmCall.stream(client, request, on_event: &MyApp.Audit.stream_event/1)

It runs before the reducer, including for finish_stream/2 output; the return value is ignored, and it is not called after the reducer halts. LlmCall.HTTP is the only HTTP transport, so raw bytes are not exposed as a hook.

A reasoning block spans several events. The text arrives incrementally, and a provider that signs its thinking blocks sends the signature in its own event, with an empty text — Anthropic on every block, Gemini on the thought parts before a function call. Keep both when you fold the stream into an assistant turn, or the block cannot be replayed in the next request:

def merge(%LlmCall.Content.Thinking{} = block, %LlmCall.StreamDelta.Thinking{} = delta) do
  %{block | text: block.text <> delta.text, signature: delta.signature || block.signature}
end

Responses over WebSocket

The Responses protocol also has a multiplexed WebSocket connection: named stream_id lanes, decoded events, and reconnect before the 60-minute limit.

{:ok, connection} = LlmCall.Dialect.OpenAI.Responses.Connection.start_link(client: client)
{:ok, stream} = LlmCall.Dialect.OpenAI.Responses.Connection.create(connection, request, stream_id: "main")

Enum.to_list(stream)

An in-flight response cannot resume across a reconnect; its stream raises LlmCall.WS.Error. The connection reconnects with exponential backoff and does not stop on failure: while disconnected, create/3 returns a :transport error and Connection.reconnect/1 retries immediately. Connection.response_id/2 returns the provider id for previous_response_id.

Tool-call fragments are assembled into complete calls by a pure helper, which never runs them:

assembler = LlmCall.StreamAssembler.new(json_decoder: &JSON.decode/1)

{:ok, assembler, calls} =
  Enum.reduce(events, {:ok, assembler, []}, fn event, {:ok, acc, calls} ->
    {:ok, acc, new_calls} = LlmCall.StreamAssembler.feed(acc, event)
    {acc, calls ++ new_calls}
  end)

Media generation

A completion can ask for image output; generated parts arrive in Response.content:

request = %LlmCall.Request{
  model: "gemini-2.5-flash-image",
  modalities: [:text, :image],
  messages: [%LlmCall.Message{role: :user, content: [%LlmCall.Content.Text{text: "a cat"}]}]
}

{:ok, response} = LlmCall.complete(client, request)
# response.content may contain %LlmCall.Content.Image{}

Dedicated endpoints take a prompt and media parameters instead of messages:

media = %LlmCall.Media.Request{
  modality: :image,
  model: "gpt-image-1",
  prompt: "a cat, watercolor",
  params: %{n: 2, size: "1024x1024"}
}

{:ok, %LlmCall.Media.Response{assets: assets}} =
  LlmCall.generate(client, media, codec: :images)

Asynchronous video returns an %LlmCall.Operation{} whose next is the request to execute; poll/3 runs it and returns the next state:

{:ok, %LlmCall.Media.Response{operation: op}} = LlmCall.generate(client, video, codec: :video)
{:ok, %LlmCall.Operation{status: :completed} = done} = LlmCall.poll(client, op, codec: :video)

next_kind: :poll means "execute next and poll"; next_kind: :asset means the body is media bytes — use LlmCall.fetch/2 for it. The library never polls, waits, or downloads on its own.

LlmCall.media_capabilities/3 reports what a codec accepts: max_n: :unsupported means no n, max_references is the reference count. A knob the codec cannot express is a validation error, never a silent drop.

Replaceable JSON layer

HTTP is Finch, one implementation. JSON is a module resolved at client-build time: the call option, the provider declaration, the compile-time application default, then Elixir's built-in JSON.

LayerModuleDefault
JSONLlmCall.JSON facadeElixir JSON
# config/config.exs of the host application
config :llm_call,
  json_library: Torque,
  capture_raw: :response,
  http_opts: [name: MyApp.Finch],
  timeout: 30_000

Torque, Jason and Poison expose tagged encode/1/decode/1; Elixir JSON exposes encode!/1/decode!/1. The LlmCall.JSON facade accepts either shape and normalizes it, so no adapter module is needed. json_library: is the single configuration key, matching Phoenix.

json_library and capture_raw are read with Application.compile_env/3; changing them requires recompiling the library. Per-client json_library:, capture_raw: (through LlmCall.Config.new/1) and provider declarations still override them without touching the compiled default. An explicit %LlmCall.Config{} uses the compiled default for both.

LlmCall.Application starts a Finch pool named LlmCall.Finch; pass http_opts: [name: MyPool] to use your own. If that name is already taken when :llm_call starts, the existing pool is used and no second one is started. http_opts accepts only :name, :pool_timeout, :request_timeout and :pool_strategy; timeout: is the read timeout, and receive_timeout in http_opts is rejected instead of ignored.

timeout: is Finch's :receive_timeout: each body chunk must arrive within it, not the whole call, so a large response on a slow link can outlive it. Add http_opts: [request_timeout: ...] when you want a wall-clock cap.

Tools

Tool data is wire data only: LlmCall.ToolSpec, LlmCall.ToolCall and LlmCall.ToolResult. The library does not derive schemas or dispatch calls — the tool_call package does that well, so use it directly and map its provider-neutral descriptions into canonical specs:

defmodule MyApp.Tools.Weather do
  use ToolCall

  @doc """
  Get the forecast for a city.

  ## Parameters
    * `city` - city name
  """
  @spec get_weather(city :: String.t()) :: {:ok, %{temp: float()}}
  def get_weather(city), do: {:ok, %{temp: 21.5}}
end

tools =
  for tool <- ToolCall.describe(MyApp.Tools.Weather) do
    %LlmCall.ToolSpec{
      name: tool.name,
      description: tool.description,
      parameters: tool.parameters
    }
  end

request = %{request | tools: tools}
{:ok, response} = LlmCall.complete(client, request)

for call <- LlmCall.Response.tool_calls(response) do
  {:ok, result} = ToolCall.run(MyApp.Tools.Weather, String.to_existing_atom(call.name), call.arguments)
  # build the %LlmCall.ToolResult{} and send the next request yourself
end

Approvals, audits and timeouts stay in tool_call, and the follow-up request stays with you: the library does not run an agent loop.

What a decoded turn can be sent back as depends on the protocol: text and tool calls always, thinking blocks and Content.Raw only where the request shape allows it. docs/design.md §8.2 has the table, and LlmCall.Codec.Conformance.assert_replays/3 pins it for an adapter.

Provider-hosted tools (web search, grounding, server tools) are builtin tools whose name is the vendor's own identifier; the codec forwards them verbatim when it declares builtin_tools: :passthrough:

%LlmCall.ToolSpec{kind: :builtin, name: "web_search_20250305", provider_options: %{"max_uses" => 5}}

Dialect adapters

Four reference adapters — the dialect families the IR is written against:

DialectModule
OpenAI Chat CompletionsLlmCall.Dialect.OpenAI.Chat
OpenAI ResponsesLlmCall.Dialect.OpenAI.Responses
Anthropic MessagesLlmCall.Dialect.Anthropic.Messages
Gemini GenerateContentLlmCall.Dialect.Gemini.GenerateContent

And four vendored media adapters, which ship with the library but are not part of the stable API surface (see LlmCall.Media):

DialectModule
OpenAI ImagesLlmCall.Media.Dialect.OpenAI.Images
Gemini Imagen predictLlmCall.Media.Dialect.Gemini.Predict
Seedance (BytePlus ModelArk)LlmCall.Media.Dialect.Seedance.Video
MiniMax HailuoLlmCall.Media.Dialect.MiniMax.Video

The library does not promise provider coverage. An adapter outside this repository is a module that uses LlmCall.Dialect.Adapter (or LlmCall.Media.Dialect.Adapter) and supplies @path, @headers, a private body/2, and the protocol's own validate_request/2; share validation through LlmCall.Dialect.Helpers instead of reimplementing it. See CONTRIBUTING.md.

OpenAI-compatible endpoints (Groq, DeepSeek, Together, vLLM, Ollama's OpenAI surface, ...) use the OpenAI Chat codec with their own base URL and provider_options; their capabilities are not uniform.

Two levels

The one-call client is the default. The encode/decode functions stay available for callers that already own a transport:

{:ok, encoded} = LlmCall.encode(LlmCall.Dialect.Anthropic.Messages, request)
# transport: encoded.method to base_url <> encoded.path, JSON-encode encoded.body
{:ok, response} = LlmCall.decode_response(LlmCall.Dialect.Anthropic.Messages, payload)

Design

See docs/design.md for the canonical IR, the codec contract, the streaming model, the client, and the failure and compatibility policy. docs/roadmap.md records what 0.1 shipped and the order in which further operation families (embeddings, rerank, speech, transcription) would arrive.

Adapters check themselves with LlmCall.Codec.Conformance or LlmCall.Media.Conformance (test support published from lib/, not a runtime dependency). The vendored media adapters run the media suite in this repository's own mix test.