Exporting to Production

Purpose and Scope

Exporting to production is the bridge between a trained Transformers checkpoint and the runtime that will actually serve it. The repository documentation frames export as a way to keep the same model while targeting optimized runtimes, cloud inference stacks, mobile devices, or edge devices without rewriting model code for every deployment environment. In this context, serialization means converting the model into a durable artifact, such as an ONNX protobuf graph or an ExecuTorch program, that another runtime can load and execute. The deployment decision is therefore less about changing model semantics and more about matching artifact format, target hardware, and runtime capabilities.

Sources: docs/source/en/serialization.md, docs/source/ro/serialization.md

The serialization documentation has evolved across languages. The current English and Romanian pages describe a broader production-export surface that includes ExecuTorch and ONNX, while several localized pages retain the earlier ONNX-focused guide that explains why serialized formats matter, how Optimum participates, and what a successful export looks like. Read together, these pages provide a practical workflow: choose a target runtime, install the matching Optimum package, export with a command line tool or programmatic API, validate the generated artifact, and then load it through a runtime-specific interface for serving or further optimization.

Sources: docs/source/ar/serialization.md, docs/source/ja/serialization.md, docs/source/ko/serialization.md, docs/source/zh/serialization.md

Relevant Source Files

• docs/source/en/serialization.md — Main English production export page covering ExecuTorch, ONNX, installation commands, CLI usage, local-model export, and deployment through ONNX Runtime-compatible tooling.
• docs/source/ro/serialization.md — Romanian version of the current production export page, confirming the same ExecuTorch and ONNX concepts and commands in the localized documentation surface.
• docs/source/ar/serialization.md — Arabic ONNX export guide that explains Optimum exporters, ONNX as an intermediate representation, CLI export, validation logs, and local checkpoint handling.
• docs/source/ja/serialization.md — Japanese ONNX guide that reinforces the concepts of standardized operators, graph optimization, quantization, ORT model classes, and pipeline compatibility.
• docs/source/ko/serialization.md — Korean ONNX guide that documents the same production flow and validation expectations for exported artifacts.
• docs/source/zh/serialization.md — Chinese ONNX guide that includes local-model export with a task argument, support for accelerators, and ONNX Runtime usage framing.
• docs/source/ar/gguf.md — GGUF interoperability page used here for the GGUF-specific production artifact discussion, including loading via gguf_file and converting back for ggml ecosystems.

Export Targets and When to Use Them

Use ExecuTorch when the production target is mobile or edge hardware that benefits from a lightweight runtime and a precomputed execution plan. The documentation describes ExecuTorch as a path that turns a PyTorch model into a graph of standardized operators, compiles that graph into an ExecuTorch program, and executes it on the target device. That makes it a deployment option for teams that want to start from a Transformers checkpoint but end with an artifact designed for constrained devices rather than a Python server process. The example export targets text generation and includes recipe, attention, cache, and quantization-related options.

Sources: docs/source/en/serialization.md, docs/source/ro/serialization.md

Use ONNX when the deployment target is an optimized runtime ecosystem rather than a single framework process. The docs define ONNX as a shared language for models, represented as a graph of standardized operators with well-defined types, shapes, and metadata. Older localized guides add that this graph is often treated as an intermediate representation that can move between frameworks and serve as input to accelerators, optimization passes, and quantization tooling. In practice, ONNX is the production format to consider when you need ONNX Runtime, TensorRT, OpenVINO, or another ONNX-compatible backend to own inference.

Sources: docs/source/en/serialization.md, docs/source/zh/serialization.md

GGUF is a different kind of deployment artifact. The supplied GGUF documentation describes it as a single-file format commonly used by GGML-based projects such as llama.cpp and whisper.cpp, with configuration attributes, tokenizer vocabulary, metadata, and tensors commonly living together in one file. Transformers support is positioned as interoperability rather than the final serving runtime: load a GGUF file into Transformers, dequantize it for PyTorch use, optionally fine-tune or inspect it, save the model and tokenizer, and convert back to GGUF with the llama.cpp conversion script for use in the GGML ecosystem.

Sources: docs/source/ar/gguf.md

Execution Flow

A typical ONNX export starts by installing the dedicated Optimum ONNX package, selecting either a Hub checkpoint or a local model directory, and running the Optimum CLI with an output directory. For Hub checkpoints, the model identifier is passed directly. For local exports, the documentation emphasizes saving the model weights and tokenizer files in the same directory first, then passing that directory as the model value. Supplying the task is important when the export must preserve a task-specific head, such as question answering or text generation; otherwise the system can infer the task or export an architecture without a task-specific head.

Sources: docs/source/en/serialization.md, docs/source/ar/serialization.md, docs/source/zh/serialization.md

uv pip install optimum-onnx
optimum-cli export onnx --model Qwen/Qwen3-8B Qwen/Qwen3-8b-onnx/
optimum-cli export onnx --model path/to/local/model --task text-generation Qwen/Qwen3-8b-onnx/
optimum-cli export onnx --help

The validation step matters because export is not only file conversion. The localized ONNX guides show logs that compare output names and tensor values against a reference model, including checks for question-answering outputs such as start and end logits. A successful export reports where the model artifact was saved, commonly as a model file inside the output directory. Treat those logs as part of the production handoff: they confirm that the serialized graph is structurally aligned with the original model and that representative outputs remain close enough for the intended tolerance before the artifact is moved into a serving runtime.

Sources: docs/source/ar/serialization.md, docs/source/ja/serialization.md, docs/source/ko/serialization.md, docs/source/zh/serialization.md

ExecuTorch follows the same high-level pattern but targets a different packaging result. The documented flow installs Optimum ExecuTorch from source, then calls the export command with a model, task, recipe, attention and cache flags, quantization choices, and an output directory. Those arguments are not incidental: they encode deployment decisions that affect the generated program and the performance profile on the target device. Teams should therefore prototype the command with the documented help output, record the final options in build automation, and treat the generated directory as a versioned deployment artifact tied to a specific model revision and runtime stack.

Sources: docs/source/en/serialization.md, docs/source/ro/serialization.md

git clone https://github.com/huggingface/optimum-executorch.git
cd optimum-executorch
pip install '.[dev]'
optimum-cli export executorch --help

API and Command Reference

The command-line surface documented for production export is intentionally compact. For ONNX, the essential parameters are the export subcommand, the model identifier or local directory, an optional task, and the output directory. For ExecuTorch, the example adds deployment-specific controls such as a recipe, custom scaled dot-product attention, custom key-value cache support, linear and embedding quantization choices, and an output directory. The docs direct readers to help commands for the full option set, which is the right source of truth because supported switches depend on the installed Optimum exporter package and backend versions.

Sources: docs/source/en/serialization.md, docs/source/ro/serialization.md

Programmatic deployment after ONNX export uses Optimum runtime classes rather than raw Transformers model classes. The English and Romanian pages show the pattern of pairing a Transformers tokenizer with an Optimum ONNX Runtime model class, while the older localized pages explain that ORTModelForXXX classes follow the AutoModel-style interface Transformers users already know. This is an important source-to-code mapping for application developers: preprocessing and tokenization can remain familiar, while model execution is delegated to the ONNX Runtime-backed class that knows how to load and run the exported graph.

Sources: docs/source/en/serialization.md, docs/source/ro/serialization.md, docs/source/ja/serialization.md, docs/source/zh/serialization.md

Deployment Notes and Next Steps

For production, choose the export path from the serving environment backward. If the system is a Python service that needs accelerator-friendly inference, ONNX plus ONNX Runtime or another ONNX runtime is a natural first option. If the system is mobile or edge, ExecuTorch is designed for that runtime model. If the artifact must interoperate with llama.cpp or another GGML-based stack, use the GGUF workflow and be aware that Transformers currently loads GGUF for PyTorch-side use by dequantizing the model first. Each path should be tested with representative inputs, pinned dependencies, and recorded export commands.

Sources: docs/source/en/serialization.md, docs/source/ar/gguf.md

After exporting, verify three things before handing the artifact to a serving team. First, confirm that the generated files live in the expected output directory and are associated with the correct checkpoint revision. Second, confirm that validation logs or runtime smoke tests show outputs compatible with the original model for the task being served. Third, document the runtime contract: tokenizer files, task name, quantization settings, cache behavior, and expected runtime class or engine. For adjacent workflows, read the quantization pages when optimizing model size, the inference optimization page when tuning latency, and the Auto Classes page when deciding how checkpoints are loaded before export.

Sources: docs/source/ar/serialization.md, docs/source/zh/serialization.md