Provider API

Purpose and Scope

The Provider API is the runtime extension point for deciding how Flue reaches language-model backends. In Flue, agents and operations select models with model specifiers such as anthropic/claude-sonnet-4-6; the provider ID is the prefix before the slash. The Provider API lets an application register that provider ID, override catalog transport settings, or introduce a provider that is not already known to Flue. This keeps model selection in agent definitions stable while allowing deployment-specific routing, authentication, gateways, and local model endpoints to be configured at runtime.

Sources: apps/docs/src/content/docs/api/provider-api.md

Use this API when the default model catalog behavior is close to what you need but the transport path is different. A common case is routing a built-in provider through an internal gateway: the model remains an Anthropic model for catalog metadata and observability semantics, but requests go to your gateway URL with your gateway credential. Another case is registering a local OpenAI-compatible endpoint such as Ollama, where Flue cannot infer the provider’s protocol or endpoint from the catalog and therefore needs explicit api and baseUrl settings.

The documentation distinguishes provider registration identity from semantic provider naming. request.providerId preserves the registered ID used by model specifiers, while request.providerName is the semantic name used by observability integrations. That distinction matters when a catalog provider is routed through a gateway: the application still selects anthropic/..., telemetry can retain the configured registration identity, and observability tooling can still reason about the underlying provider family. The Provider API therefore sits between author-facing model strings and the lower-level HTTP or platform binding used to make the request.

Relevant Source Files

• apps/docs/src/content/docs/api/provider-api.md - Defines the public Provider API documentation, including imports from @flue/runtime, the registerProvider() signature, provider registration behavior, HTTP registration fields, and examples for gateway and custom provider registration.

API Components

The public imports documented for this page come from @flue/runtime. The documented import set includes registerProvider, registerApiProvider, HttpProviderRegistration, and ProviderRegistration. registerProvider() is the primary entry point for binding a provider ID to runtime transport configuration. registerApiProvider() appears as the companion extension point for API protocol slugs: HttpProviderRegistration.api can use a built-in API slug or one registered through registerApiProvider(). Type imports describe the supported registration shapes rather than creating runtime values.

Sources: apps/docs/src/content/docs/api/provider-api.md

import {
  registerApiProvider,
  registerProvider,
  type HttpProviderRegistration,
  type ProviderRegistration,
} from '@flue/runtime';

registerProvider()

function registerProvider(providerId: string, registration: ProviderRegistration): void;

registerProvider() registers or replaces the settings for one provider ID. Calls do not accumulate: the latest call for a provider ID is the effective override. When the ID is already present in Flue’s catalog, the catalog remains the baseline. That means metadata such as context window, cost, model information, and wire protocol can still come from the catalog, while the registration layers on deployment-specific options such as baseUrl, apiKey, headers, default token limits, or response-storage behavior. For a catalog provider, this makes override registration concise and less error-prone than redefining every model from scratch.

registerProvider('anthropic', {
  baseUrl: 'https://gateway.example.com/anthropic',
  apiKey: process.env.GATEWAY_KEY,
});

For provider IDs that are not in the catalog, the registration must supply enough information for Flue to make requests. The documented requirements are api and baseUrl: api tells Flue which wire protocol to use, while baseUrl tells it where to send requests. Once registered, the provider ID becomes usable in model specifiers for agents and operations. The example in the docs registers ollama with an OpenAI completions-compatible protocol and a local endpoint, making specifiers such as ollama/llama3.1:8b valid in the application.

registerProvider('ollama', {
  api: 'openai-completions',
  baseUrl: 'http://localhost:11434/v1',
});

Registration Types Reference

ProviderRegistration is documented as a union of HttpProviderRegistration and CloudflareAIBindingRegistration. HTTP registrations are the ordinary URL-backed form and are the most portable configuration shape across Node and gateway deployments. Cloudflare AI binding registrations are Cloudflare-specific and are used when model access is provided by a Workers AI binding instead of a direct HTTP endpoint. The union is important because it keeps the provider registry focused on model connection paths while allowing target-specific runtime integrations to participate through the same model-specifier mechanism.

Sources: apps/docs/src/content/docs/api/provider-api.md

type ProviderRegistration = HttpProviderRegistration | CloudflareAIBindingRegistration;

HttpProviderRegistration contains both provider-wide defaults and model-specific overrides. api selects the wire protocol, baseUrl selects the endpoint root, apiKey and headers configure authentication and request metadata, and contextWindow and maxTokens provide default capacity information for models resolved through the registration. The models map lets a provider give per-model overrides for contextWindow and maxTokens, which is useful when several models share the same endpoint but have different limits. storeResponses controls whether provider response-storage behavior is enabled for the registration when supported by the underlying integration.

interface HttpProviderRegistration {
  api?: Api;
  baseUrl?: string;
  apiKey?: string;
  headers?: Record<string, string>;
  contextWindow?: number;
  maxTokens?: number;
  models?: Record<
    string,
    {
      contextWindow?: number;
      maxTokens?: number;
    }
  >;
  storeResponses?: boolean;
}

System-to-Code Mapping

Provider registration is intentionally small because most model-facing code should not know about transport details. Agent definitions can continue to say model: 'anthropic/claude-sonnet-4-6' or model: 'ollama/llama3.1:8b'; the provider registry determines how the provider prefix resolves when the runtime needs to call the model. This separation lets application code keep durable agent behavior, tools, skills, and workflows independent from infrastructure concerns such as gateway URLs, secret sources, and deployment target.

Sources: apps/docs/src/content/docs/api/provider-api.md

The documented layering rules are the key operational constraint. For catalog providers, the effective registration is catalog defaults plus the latest call’s options. Headers merge by key over catalog model headers, while other configured values override defaults at the provider or model level. For unknown providers, there is no catalog baseline, so the registration must be complete enough to choose a protocol and endpoint. Because repeated calls replace previous settings, applications should centralize provider registration during startup or configuration loading instead of scattering partial overrides across modules.

Usage Guidance and Next Steps

Register providers before agents or workflows are invoked so model specifiers resolve predictably during execution. Prefer catalog provider IDs when the underlying model family is already known to Flue, because that preserves catalog metadata and reduces the amount of configuration you must maintain. Use a custom provider ID when you are adding a local endpoint, an internal model service, or another backend that Flue cannot infer. When introducing custom API protocols, pair HttpProviderRegistration.api with the documented registerApiProvider() extension point so the provider registry can resolve both the endpoint and request format.

For broader model selection, authentication patterns, and Workers AI examples, continue to the Models page. For agent-facing usage, read the Agent API and Building Agents pages to see where model specifiers are placed in definitions and profiles. For telemetry behavior, pair this page with Observability so the distinction between providerId and providerName is reflected correctly in exported traces, metrics, or logs.