Audio reference
The audio surface in this SDK covers request-based speech generation, speech-to-text transcription, and speech translation. It is the right reference when an application sends a bounded request, such as a text prompt that should become spoken output, or an audio file that should become text. For live voice agents, live captions, and very low-latency partial transcripts, the first-party platform docs distinguish realtime sessions from these request-response APIs. In this repository, the request-based audio namespace is exposed through the generated client resource tree, so callers reach it from the OpenAI client rather than constructing endpoint URLs manually.
Sources: src/resources/audio/index.ts, tests/api-resources/audio/speech.test.ts, tests/api-resources/audio/translations.test.ts
The audio namespace is generated from the OpenAPI specification, and the SDK exports both resource classes and TypeScript types for the public contract. That means the main developer experience is strongly typed request objects and strongly typed response objects, with the generated reference in the repository acting as the complete method catalog. The tests shown here exercise the same public client shape that user code sees: instantiate the OpenAI client, call a method under the audio namespace, and await either the parsed result or one of the raw-response helper forms when HTTP response metadata is needed.
Sources: api.md, src/resources/audio/index.ts, tests/api-resources/audio/translations.test.ts
Purpose and Scope
Use this page as a method-level map for the three primary request-based audio resource families. Speech generation turns text into audio. Transcriptions turn uploaded audio into text and, depending on request options and model behavior, may support richer result shapes such as verbose data, words, segments, diarized segments, or streaming events. Translations accept an audio file and produce translated text output. The page focuses on the public SDK names, request fields demonstrated by tests, exported response types, and practical differences between the resource families. It does not replace model-selection guidance; instead, it shows how those model choices attach to SDK calls.
Official platform guidance separates audio tasks by modality. Audio input means the model receives sound from a user or application; audio output means the API returns spoken audio; text transcripts are generated from speech; and text prompts can control generated speech. This SDK page maps those concepts to the request-response client: text prompt plus voice becomes speech output, audio file plus transcription model becomes text, and audio file plus translation model becomes translated text. If the product requirement is continuous interaction rather than a bounded file or bounded speech generation request, the related realtime pages are the better next stop.
Relevant Source Files
- api.md — Generated SDK API reference for the OpenAI TypeScript and JavaScript library, including the audio resource methods and exported types used by application code.
- src/resources/audio/index.ts — Barrel export for the generated audio namespace, including resource classes and audio-related TypeScript types.
- tests/api-resources/audio/speech.test.ts — API resource test showing the speech creation call and its required and optional request fields.
- tests/api-resources/audio/translations.test.ts — API resource tests showing translation creation with file upload input, optional parameters, and raw-response helper usage.
System-to-Code Mapping
The file that anchors the namespace is the audio resource index. It re-exports the top-level audio resource, shared audio model and response-format types, and the concrete resource classes for speech, transcriptions, and translations. From a user perspective, this explains why the client groups calls as audio operations instead of separate top-level services. From a maintainer perspective, it also identifies where generated type names enter the package export graph. When TypeScript users import types for a wrapper function, these are the names that represent the generated audio contract rather than hand-written local interfaces.
Sources: src/resources/audio/index.ts
The speech family is represented by a generated Speech resource and its associated model and create-parameter types. The resource test demonstrates that the public method accepts text input, a speech model, and a voice, with optional instructions, response format, speed, and stream format. Those fields describe the core degrees of freedom for text-to-speech: what should be spoken, what model should generate it, which voice should render it, and how the resulting audio should be delivered. The test is skipped because binary tests are marked broken in that suite, but it still records the intended public request shape for the generated client.
Sources: tests/api-resources/audio/speech.test.ts, src/resources/audio/index.ts
The transcriptions family is exposed as a generated Transcriptions resource with several response and event-oriented type exports. The index lists plain transcription results, verbose transcription results, diarized results, segment and word types, include options, and streaming event types such as text deltas, text completion, and text segments. Even without inspecting implementation internals, those exports show that transcription is not only a single plain text shape. Callers can type code for simple responses, richer time-aligned data, and streaming-oriented event handling when the selected endpoint and parameters support those forms.
Sources: src/resources/audio/index.ts, api.md
The translations family is represented by a generated Translations resource with plain and verbose response types plus a create-parameter type. The translation tests show the required request pattern: provide a file object and a model. They also show optional prompt, response format, and temperature fields. In practice, that makes translation structurally closer to transcription than speech generation because it starts from an uploaded audio file rather than a text prompt. The output, however, is translated text rather than a transcript in the source language, so application pipelines should keep the two tasks distinct.
Sources: tests/api-resources/audio/translations.test.ts, src/resources/audio/index.ts
Method Reference
| Resource | Method | Required request shape shown in source | Optional request fields shown in source | Response handling shown in source |
|---|---|---|---|---|
| Speech | client.audio.speech.create(...) | input, model, voice | instructions, response_format, speed, stream_format | Await the method result, which is an audio response body rather than a JSON transcript in the tested scenario. |
| Transcriptions | client.audio.transcriptions.create(...) | Generated create params type is exported; callers provide audio input according to the generated API reference. | Generated non-streaming and streaming parameter types are exported, along with include and event types. | Use the generated response types for plain, verbose, diarized, or streaming transcription workflows. |
| Translations | client.audio.translations.create(...) | file, model | prompt, response_format, temperature | Await parsed data, or use raw-response helpers such as asResponse() and withResponse(). |
The speech create call is the audio method to reach for when the application already has text and needs spoken output. The required fields in the resource test are intentionally small: input text, model, and voice. Optional instructions let the caller influence delivery, while response format and stream format describe output transport or encoding preferences. Speed is also represented as a request field in the test, so wrappers that expose user-controlled playback style should validate and forward it deliberately rather than hiding it in unrelated application settings.
Sources: tests/api-resources/audio/speech.test.ts
The translation create call is the clearest test-backed example of file upload handling on this page. The test imports the SDK file helper, turns a buffer into a named file, and passes that file to the translation method with the model. This pattern matters because browser, Node, and serverless runtimes can represent file-like inputs differently. Application code should prepare the audio as a supported file object before calling the generated method, then keep prompts, response formatting, and temperature adjustments close to the translation call so request behavior remains auditable.
Sources: tests/api-resources/audio/translations.test.ts
Response Handling and Raw HTTP Access
Most users can await the audio method and work with the parsed SDK result. The translation tests also demonstrate the lower-level response helpers that are available on the returned promise-like object. Calling the raw response helper yields a Response instance, awaiting the original operation yields parsed data, and the combined helper returns both the parsed data and raw response together. This pattern is useful when a production service needs response headers, status information, tracing integration, or logging metadata while still preserving the typed parsed object used by business logic.
Sources: tests/api-resources/audio/translations.test.ts
This distinction is especially important for audio workflows because the returned payload may not always be ordinary JSON. Speech generation can return binary audio data or a stream-like body, while transcription and translation commonly return text-oriented objects. A wrapper library should avoid assuming that every audio call can be handled by the same serializer. Instead, model the branch explicitly: speech output goes through audio storage or playback code, while transcription and translation outputs go into text processing, search indexing, captioning, analytics, or review workflows.
Sources: tests/api-resources/audio/speech.test.ts, tests/api-resources/audio/translations.test.ts
Type Exports and Public Contract
The audio index exports shared type names as part of the SDK's public surface. For speech, the relevant names are Speech, SpeechModel, and SpeechCreateParams. For transcriptions, the exported names include Transcription, TranscriptionVerbose, TranscriptionDiarized, TranscriptionSegment, TranscriptionWord, TranscriptionCreateResponse, TranscriptionCreateParams, and separate non-streaming and streaming parameter types. For translations, the exported names include Translation, TranslationVerbose, TranslationCreateResponse, and TranslationCreateParams. These exports allow application authors to type helper functions without reaching into generated implementation files.
Sources: src/resources/audio/index.ts
For maintainers, the export list is also a compatibility signal. A generated resource or type that appears in this index is part of what downstream TypeScript projects may import. Renaming, removing, or narrowing these names would affect code that builds abstractions around audio workflows. When adding a service wrapper, prefer importing the SDK's generated request and response types over duplicating a subset of fields. That keeps local code aligned with future OpenAPI-driven changes and makes resource tests a stronger signal when the generator updates request shapes.
Sources: src/resources/audio/index.ts, api.md
Implementation Notes and Edge Cases
Treat request-based audio APIs and realtime audio APIs as different architecture choices. Official guidance recommends realtime sessions for low-latency voice agents, continuous translation, and live transcript deltas, while request-based audio APIs fit files, bounded audio requests, and generated speech that does not require an open session. In SDK terms, this page covers the audio resource methods under the standard client namespace. If an application needs persistent bidirectional audio, interruption handling, tool calls during speech, or continuously emitted transcript deltas, use the realtime reference and streaming-event guidance alongside this page.
The test suite also hints at a practical edge case: binary speech testing is skipped in the generated resource test. That does not remove the public speech method, but it is a reminder to validate audio output in the runtime where the application will run. Node services, browsers, workers, and edge runtimes differ in file APIs, streams, and filesystem access. Keep generated speech handling isolated behind a small adapter so the application can write a file in Node, return a response body from an HTTP route, or hand bytes to a browser audio element without changing the API call itself.
Sources: tests/api-resources/audio/speech.test.ts
Next Steps
Start with the simplest matching method: use speech creation for text-to-speech, transcription creation for speech-to-text, and translation creation for speech translation from an uploaded audio file. Then decide whether parsed data is sufficient or whether the service also needs raw HTTP response access. For production wrappers, import the generated parameter and response types from the SDK, centralize file preparation, and keep runtime-specific stream or binary handling outside the resource call. Read the broader audio guide for task selection and the realtime reference when the product requirement shifts from bounded requests to live audio sessions.