Community Integrations

Purpose and Scope

Community integrations document how the Transformers ecosystem connects to specialized training frameworks, inference servers, local runtimes, and hardware-specific backends. The reader problem is practical: a model may be loaded and fine-tuned with Transformers APIs, but production or research workflows often need higher throughput, lower memory use, distributed post-training, or a non-Python deployment target. These pages explain where Transformers remains the model, tokenizer, configuration, or checkpoint interface, and where the partner project takes over execution, optimization, serving, or training orchestration.

Sources: docs/source/en/community_integrations/trl.md, docs/source/en/community_integrations/vllm.md, docs/source/en/community_integrations/llama_cpp.md, docs/source/en/community_integrations/sglang.md, docs/source/en/community_integrations/mlx.md, docs/source/en/community_integrations/tensorrt-llm.md, docs/source/en/community_integrations/unsloth.md, docs/source/en/community_integrations/axolotl.md

A useful way to read this section is to separate integrations by workflow phase. TRL, Unsloth, and Axolotl are training or post-training integrations: they wrap or extend Trainer-style workflows, add reinforcement or preference-learning methods, and frequently rely on PEFT, quantization, or custom kernels. vLLM, SGLang, TensorRT-LLM, llama.cpp, and MLX are primarily inference or runtime integrations: they preserve the Hub checkpoint and configuration contract while changing how model execution is scheduled, compiled, converted, or placed on hardware. That separation helps you choose an integration before thinking about syntax.

Relevant Source Files

• docs/source/en/community_integrations/trl.md - Describes TRL as a post-training framework with SFT, GRPO, DPO, Trainer-based method trainers, and AutoConfig-based model loading.
• docs/source/en/community_integrations/vllm.md - Documents vLLM serving with model_impl="transformers", continuous batching, PagedAttention, tokenizer loading, safetensors weights, and registry fallback behavior.
• docs/source/en/community_integrations/llama_cpp.md - Explains GGUF conversion, local llama-cli and llama-server deployment, and how config, tokenizer, architecture registry, tensor mapping, and vocabulary packaging form the conversion flow.
• docs/source/en/community_integrations/sglang.md - Covers SGLang Engine and server launch with model_impl="transformers", RadixAttention routing, tensor-parallel linear replacement, and compatibility expectations for attention backends.
• docs/source/en/community_integrations/mlx.md - Shows MLX and mlx-lm usage for Apple silicon and CUDA, safetensors-based loading, model-tokenizer return values, and bidirectional MLX weight compatibility.
• docs/source/en/community_integrations/tensorrt-llm.md - Describes AutoDeploy for NVIDIA GPUs, TensorRT engine compilation, in-flight batching, paged KV caching, tensor parallelism, torch.export graph extraction, and attention kernel replacement.
• docs/source/en/community_integrations/unsloth.md - Documents Unsloth fast fine-tuning, 4-bit, 8-bit, and 16-bit loading, FastLanguageModel, LoRA target modules, UnslothTrainer, and Trainer patching.
• docs/source/en/community_integrations/axolotl.md - Documents YAML-driven Axolotl training, ModelLoader, AutoModel class selection, device maps, quantization config, attention backend setup, PEFT, bitsandbytes, and AxolotlTrainer.

Integration Categories

For post-training, TRL is the most direct extension of Transformers training vocabulary. Its docs define TRL as a framework for foundation-model post-training with methods such as supervised fine-tuning, group relative policy optimization, and direct preference optimization. Method-specific trainers build on Trainer, while configuration classes extend TrainingArguments with fields specific to the optimization method. Model loading still begins with AutoConfig.from_pretrained and then instantiates the configured model class through from_pretrained, so the Transformers checkpoint contract remains the foundation even when the loss, reward scoring, or generation loop is owned by TRL.

Sources: docs/source/en/community_integrations/trl.md

Unsloth and Axolotl also focus on training, but they make different tradeoffs. Unsloth wraps Transformers APIs through FastLanguageModel.from_pretrained, loads a base model with AutoModelForCausalLM.from_pretrained, and patches attention, decoder, rotary embedding, compute_loss, and training_step behavior for speed or compatibility. Axolotl is configuration-first: a YAML file names base_model, model_type, tokenizer_type, datasets, sequence length, batch settings, and optimization values before axolotl train launches the run. Its ModelLoader prepares device maps, quantization, attention backend selection, model-class resolution, PEFT adapters, bitsandbytes, and trainer mixins around the standard Transformers loading and training loop.

Sources: docs/source/en/community_integrations/unsloth.md, docs/source/en/community_integrations/axolotl.md

For server-side inference, vLLM and SGLang both expose an explicit Transformers backend switch. vLLM uses model_impl="transformers" in Python or --model-impl transformers in vllm serve, while SGLang uses the same value in sgl.Engine or sglang.launch_server. In both cases, the integration keeps the Transformers model structure available but lets the serving engine optimize execution. vLLM emphasizes continuous batching, compact KV cache memory through PagedAttention, tokenizer caching, safetensors loading, and replacement of common model modules with optimized versions. SGLang emphasizes low-latency serving, agentic workflow support, RadixAttention, tensor-parallel linear layers, and safetensors weight loading into the selected model.

Sources: docs/source/en/community_integrations/vllm.md, docs/source/en/community_integrations/sglang.md

For local or hardware-specific execution, llama.cpp, MLX, and TensorRT-LLM each adapt the same checkpoint ecosystem to a different runtime. llama.cpp converts a Transformers checkpoint into GGUF, packages weights, tokenizer, and metadata, and then runs locally with llama-cli or llama-server without Python or heavy server infrastructure. MLX runs safetensors-backed language models in an MLX nn.Module parameter tree, especially for Apple silicon shared memory workflows, without requiring weight conversion for supported architectures. TensorRT-LLM AutoDeploy starts from a Hub model id, loads Transformers configuration and model data, exports the graph, replaces attention with TensorRT-LLM kernels, and compiles an optimized NVIDIA GPU runtime.

Sources: docs/source/en/community_integrations/llama_cpp.md, docs/source/en/community_integrations/mlx.md, docs/source/en/community_integrations/tensorrt-llm.md

System-to-Code Mapping

This mapping is intentionally contract-oriented rather than package-catalog oriented. The shared contract is that a Transformers-compatible checkpoint exposes configuration, architecture metadata, tokenizer files, and named tensors in a form that other systems can inspect. From there, each integration decides whether to call the original Transformers model directly, instantiate it as a fallback, patch modules before loading, convert tensors to another format, or compile the resulting graph into a different runtime. When debugging an integration, confirm first that the model id, config, tokenizer, and safetensors files load in the expected place, then inspect the partner framework’s replacement or conversion step.

Example Flows

A TRL post-training run begins with a dataset and a method-specific trainer. The documented GRPO example loads trl-lib/DeepMath-103K, creates GRPOTrainer with model="Qwen/Qwen2-0.5B-Instruct", reward_funcs=accuracy_reward, and train_dataset=dataset, then calls trainer.train. That flow is familiar to Transformers users because the trainer boundary is still present, but the integration adds generation and reward-aware training behavior. Use this path when the task is preference learning, reward modeling, reinforcement-style optimization, or supervised fine-tuning with TRL recipes rather than a plain Trainer loop.

Sources: docs/source/en/community_integrations/trl.md

from datasets import load_dataset
from trl import GRPOTrainer
from trl.rewards import accuracy_reward
 
dataset = load_dataset("trl-lib/DeepMath-103K", split="train")
trainer = GRPOTrainer(
    model="Qwen/Qwen2-0.5B-Instruct",
    reward_funcs=accuracy_reward,
    train_dataset=dataset,
 )
trainer.train()

A vLLM or SGLang flow starts from the same model id but optimizes online generation. For vLLM, the Python LLM constructor accepts model="meta-llama/Llama-3.2-1B" and model_impl="transformers"; the CLI equivalent is vllm serve with --task generate and --model-impl transformers. For SGLang, sgl.Engine accepts a model path and model_impl="transformers", while python3 -m sglang.launch_server exposes host and port options. Choose these paths when request scheduling, batching, cache efficiency, or server behavior matters more than staying inside a raw generate call.

Sources: docs/source/en/community_integrations/vllm.md, docs/source/en/community_integrations/sglang.md

vllm serve meta-llama/Llama-3.2-1B \
    --task generate \
    --model-impl transformers

python3 -m sglang.launch_server \
  --model-path meta-llama/Llama-3.2-1B-Instruct \
  --model-impl transformers \
  --host 0.0.0.0 \
  --port 30000

A conversion or hardware-runtime flow changes the execution artifact. llama.cpp uses convert_hf_to_gguf.py to produce a single GGUF file and then deploys Hub GGUF models with llama-cli -hf or llama-server -hf. TensorRT-LLM uses an AutoDeploy script or LLM class to construct an optimized runtime from a model id. MLX uses mlx_lm.load to return a model and tokenizer and generate text without conversion for supported language-model architectures. These workflows are best when the target environment is local CPU-style deployment, Apple silicon shared-memory execution, CUDA through MLX, or NVIDIA TensorRT compilation.

Sources: docs/source/en/community_integrations/llama_cpp.md, docs/source/en/community_integrations/mlx.md, docs/source/en/community_integrations/tensorrt-llm.md

python3 convert_hf_to_gguf.py ./models/openai/gpt-oss-20b \
  --outfile gpt-oss-20b.gguf

from mlx_lm import load, generate
 
model, tokenizer = load("openai/gpt-oss-20b")
output = generate(model, tokenizer, prompt="The capital of France is", max_tokens=100)

Selection Guidance and Next Steps

Choose the integration by deciding which layer you want to delegate. Delegate the training loop to TRL when you need method-specific post-training semantics, to Unsloth when you need patched fast fine-tuning with low-bit loading and LoRA-style setup, and to Axolotl when a reproducible YAML training configuration and broader model-loader orchestration are preferred. Delegate the inference scheduler to vLLM or SGLang when throughput, batching, attention backends, and serving APIs dominate the problem. Delegate the runtime artifact to llama.cpp, MLX, or TensorRT-LLM when deployment environment, file format, hardware kernels, or compilation is the constraint.

Before moving to an integration, verify the model works with the corresponding lower-level Transformers primitive: AutoConfig, AutoTokenizer or a processor where applicable, and the appropriate AutoModel class. Then apply the integration’s documented switch or loader, such as model_impl="transformers", --model-impl transformers, FastLanguageModel.from_pretrained, a ModelLoader-backed Axolotl YAML file, mlx_lm.load, convert_hf_to_gguf.py, or TensorRT-LLM AutoDeploy. If the next task is model loading, read Auto Classes and Model Loading; if it is training, read Trainer, Fine-tuning, and PEFT; if it is serving, read Inference Optimization, Continuous Batching, and Exporting to Production.