Realtime API
Purpose and Scope
The Realtime API is the SDK surface for low-latency, bidirectional interactions with realtime OpenAI models and related call flows. In this repository, it is represented by generated REST resources, stable and beta WebSocket helpers, and tests that pin URL construction and message-queue behavior. Use this page when you need to understand which SDK modules participate in realtime workflows, how the SDK distinguishes model-based and call-based connection targets, and where generated types such as call parameters and client-secret responses fit into application code. Sources: api.md, src/resources/realtime/index.ts, src/beta/realtime/index.ts, tests/realtime.test.ts, tests/ws.test.ts
Realtime integrations have two different shapes that are easy to confuse. A server-to-server WebSocket connection opens a realtime session directly from trusted backend code, commonly using a normal API key. Browser or mobile clients often use WebRTC or an ephemeral credential pattern so long-lived secrets do not leave the server. The SDK repository reflects this split by exposing generated REST resources for session and call setup, while separately testing WebSocket URL construction and queued outbound message behavior. The resource layer is the typed API entry point; the WebSocket layer is the transport-oriented helper layer that gets events to and from the model.
Relevant Source Files
- api.md — Generated API reference entry for the package; use it as the broad method and type catalog when pairing this conceptual guide with exact SDK signatures.
- src/resources/realtime/index.ts — Stable generated Realtime resource barrel that exports
Calls,ClientSecrets,Realtime, and request/response types for realtime call and client-secret flows. - src/beta/realtime/index.ts — Beta Realtime entry point that exposes
OpenAIRealtimeError, preserving the beta error surface separately from the stable generated resources. - tests/realtime.test.ts — Realtime URL-builder tests covering stable and beta helpers, OpenAI versus Azure clients,
modeltargets,callIDtargets, and validation errors. - tests/ws.test.ts — WebSocket send-queue tests covering JSON messages, raw messages, byte limits, flush behavior, re-queue-on-error behavior, and drain behavior.
API Components
The stable generated Realtime resource index exports three resource families: Realtime, Calls, and ClientSecrets. Realtime is the namespace-level resource, while Calls contains call-oriented operations and typed parameter names including CallAcceptParams, CallReferParams, and CallRejectParams. ClientSecrets contains the client-secret resource and types including ClientSecretCreateParams, ClientSecretCreateResponse, RealtimeSessionCreateResponse, RealtimeTranscriptionSessionCreateResponse, and RealtimeTranscriptionSessionTurnDetection. These exports show that realtime setup is not only a WebSocket concern: applications may first create or manage REST objects, then use the resulting session, call, or secret data to establish a media or event connection. Sources: src/resources/realtime/index.ts, api.md
The beta Realtime entry point is intentionally much smaller in the provided source evidence: it exports OpenAIRealtimeError from its internal base module. That matters for migration and error handling because beta realtime helpers can have their own error class even when the stable generated resource tree is the preferred place for REST operations. The tests explicitly import both stable and beta buildRealtimeURL helpers and run many of the same URL expectations against both. Treat the beta surface as a compatibility layer where it appears in existing applications, and prefer stable helpers for newly implemented flows that need behavior validated by the stable tests. Sources: src/beta/realtime/index.ts, tests/realtime.test.ts
Compact reference
| Component | Public names visible in source | Use in realtime workflows |
|---|---|---|
| Stable namespace | Realtime | Groups generated realtime REST resources. |
| Calls resource | Calls, CallAcceptParams, CallReferParams, CallRejectParams | Represents accepting, referring, or rejecting realtime call flows. |
| Client secrets resource | ClientSecrets, ClientSecretCreateParams, ClientSecretCreateResponse | Represents server-created credentials or session setup data for client-side realtime connections. |
| Realtime session responses | RealtimeSessionCreateResponse, RealtimeTranscriptionSessionCreateResponse, RealtimeTranscriptionSessionTurnDetection | Captures typed responses for realtime and transcription session creation. |
| Beta error | OpenAIRealtimeError | Error type exposed by the beta realtime module. |
| URL helper behavior | buildRealtimeURL imported from stable and beta internal bases in tests | Builds WebSocket URLs for model or call targets and validates invalid combinations. |
| WebSocket queue | SendQueue, RawWebSocketData in tests | Buffers outbound WebSocket messages and preserves unsent data after send errors. |
Connection Model and URL Behavior
The realtime URL tests define the most concrete source-backed connection rules. For standard OpenAI clients, a model target such as gpt-realtime produces a WebSocket URL ending in /realtime?model=gpt-realtime, and the tests preserve a legacy shortcut where the model can be passed as a string rather than an object. A sideband OpenAI connection uses a call target instead, producing /realtime?call_id=rtc_123. The validation rule is strict: callers must pass exactly one of model or callID when opening a Realtime WebSocket, and the tests assert that both missing and multiple targets throw the same explanatory error. Sources: tests/realtime.test.ts
Azure OpenAI has its own realtime URL rules in the test suite. When the Azure client uses a deployment-style model target, the URL includes api-version and deployment query parameters. For callID connections, the stable helper normalizes Azure clients to a GA-style /openai/v1/realtime?call_id=rtc_123 URL across clients configured with an API version, a v1 base URL, or an endpoint. The beta helper deliberately rejects Azure callID connections and tells users that Azure callID connections require the stable Realtime helpers. These tests are important deployment signals because they prevent regressions in base URL handling, Azure endpoint normalization, and stable-versus-beta behavior. Sources: tests/realtime.test.ts
Official Realtime documentation adds the operational context around these SDK behaviors. WebSockets are a strong fit for server-to-server realtime integrations because the API key remains on the backend, while WebRTC is usually recommended for browser and mobile clients because it is better suited to client-side media and peer-connection scenarios. SIP call flows add another entry path: a telephony provider can route calls to OpenAI, which triggers webhook events that an application can accept or reject by call identifier. In SDK terms, those call identifiers line up with the call-target URL behavior and the Calls resource family rather than a plain model-only session.
WebSocket Message Queue Semantics
The WebSocket tests focus on SendQueue, a small but important primitive for reliable realtime sending. The queue accepts structured JSON messages through enqueue and raw data through enqueueRaw, tracking a byte limit for the queued payloads. The tests show a practical rule: a single oversized message is accepted when the queue is empty, but once there is already queued data, adding a message that would exceed the configured limit returns false. That gives the transport a way to avoid unbounded buffering while still allowing one large event to move through the system. Sources: tests/ws.test.ts
Flush and recovery behavior are also tested. flush serializes structured messages with JSON.stringify, sends raw messages as raw data, and resets the queue after successful delivery. If the send callback throws partway through a flush, the failed message and all later messages are re-queued, so callers can inspect or retry the unsent portion rather than silently dropping events. drain returns unsent messages and resets the queue, preserving enough type information to distinguish { type: 'message', message: ... } from { type: 'raw', data: ... }. For realtime applications, this behavior is part of backpressure and reconnect hygiene: when a socket is not writable, the SDK can keep a bounded record of what still needs to be sent. Sources: tests/ws.test.ts
Implementation and Testing Signals
A good way to read the realtime implementation is to separate generated API coverage from transport safeguards. Generated resources in src/resources/realtime/index.ts provide typed REST access for calls and client secrets. The beta module provides a beta-specific error export. The realtime tests verify that URL generation respects client configuration, stable and beta helper differences, and Azure-specific deployment rules. The WebSocket tests verify that queued outbound data is bounded, flushed in order, and recoverable after a send error. Together, these files describe an SDK surface that combines generated OpenAPI types with hand-tested connection behavior. Sources: src/resources/realtime/index.ts, src/beta/realtime/index.ts, tests/realtime.test.ts, tests/ws.test.ts
When building with this SDK, start by deciding which connection mechanism belongs in your architecture. Use backend WebSockets when your server can hold the API key and you need event-level bidirectional control. Use browser-oriented WebRTC patterns when the client handles media directly, and use the REST client-secret or session setup resources to keep credentials short-lived. Use call-oriented flows when a SIP or sideband integration gives you a callID. Then validate the target shape before opening the connection: model sessions and call sessions are mutually exclusive in the tested helper behavior. For adjacent details, read the pages on client configuration and authentication, streaming and events, webhooks, Azure OpenAI, and deployment and data controls.