Quickstart

Purpose and Scope

This quickstart gives you the shortest path from a working Transformers installation to real inference. The examples in the selected source files show the two main usage levels: a high-level pipeline that chooses preprocessing and postprocessing for a task, and lower-level AutoTokenizer, AutoProcessor, and model classes that expose the model input tensors and generate() call directly. The page uses Qwen-family documentation because those pages demonstrate modern text, audio, and multimodal workflows with the same public Transformers patterns used throughout the library.

Sources: docs/source/en/model_doc/qwen3_5.md, docs/source/en/model_doc/qwen3_5_moe.md, docs/source/en/model_doc/qwen3_asr.md

A useful mental model is that a checkpoint on the Hugging Face Hub contains weights and configuration for a model architecture, while Transformers supplies Python classes that know how to load that checkpoint. In the source examples, checkpoints such as Qwen/Qwen3.5-9B, Qwen/Qwen3.5-35B-A3B, Qwen/Qwen3-30B-A3B, Qwen/Qwen3-Next-80B-A3B-Instruct, and Qwen/Qwen3-ASR-1.7B-hf are loaded with from_pretrained(). The same call downloads model assets, constructs the appropriate configuration, and places weights into the selected class.

Relevant Source Files

• docs/source/en/model_doc/qwen3_5_moe.md - demonstrates a text-generation pipeline and direct Qwen3_5MoeForCausalLM loading for the sparse Qwen3.5 MoE family, plus router-logit and long-context notes.
• docs/source/en/model_doc/qwen3_5.md - demonstrates dense Qwen3.5 text generation with both pipeline and Qwen3_5ForCausalLM, including device_map="auto" usage.
• docs/source/en/model_doc/qwen3_asr.md - demonstrates automatic speech recognition with AutoProcessor, AutoModelForMultimodalLM, and processor helpers for transcription requests.
• docs/source/en/model_doc/qwen3_moe.md - demonstrates the older Qwen3 MoE quickstart using pipeline and AutoModelForCausalLM for causal language modeling.
• docs/source/en/model_doc/qwen3_next.md - demonstrates chat-style prompt formatting with tokenizer.apply_chat_template() before calling generate().
• docs/source/en/model_doc/qwen3_omni_moe.md - documents multimodal Qwen3-Omni-MOE usage constraints, model class choices for text and audio output, and processor chat-template behavior.

Core Primitives

Start with pipeline when you want the fastest working inference call. A pipeline is a task-level wrapper: you provide a task name such as text-generation, a checkpoint name, and optional placement arguments like device_map="auto"; the wrapper handles tokenization, model invocation, and response formatting. The Qwen3.5 and Qwen3.5 MoE pages both show this pattern, returning a list whose first element contains generated text under the generated_text key. This is the quickest way to validate installation, checkpoint access, and accelerator placement before writing lower-level application code.

Sources: docs/source/en/model_doc/qwen3_5.md, docs/source/en/model_doc/qwen3_5_moe.md, docs/source/en/model_doc/qwen3_moe.md

Use the lower-level API when you need control over prompts, tensors, decoding, or multimodal inputs. For text-only causal language modeling, the common pair is AutoTokenizer.from_pretrained() or a model-specific tokenizer plus AutoModelForCausalLM.from_pretrained() or a model-specific causal LM class. The tokenizer converts strings into tensors with return_tensors="pt"; moving those tensors to model.device keeps input placement consistent with device_map="auto". The model then generates token IDs with generate(), and the tokenizer decodes those IDs back into text.

For processor-based models, replace the tokenizer-only step with an AutoProcessor. A processor is the front door for inputs that are not just plain text, such as audio, images, video, or mixed chat messages. The Qwen3 ASR page uses processor.apply_transcription_request() to format audio for speech recognition, then calls AutoModelForMultimodalLM.from_pretrained() and generate(). The Qwen3-Omni-MOE notes similarly state that its processor has an apply_chat_template() method for converting chat messages into model inputs, which is the multimodal counterpart to tokenizer chat templating.

Sources: docs/source/en/model_doc/qwen3_asr.md, docs/source/en/model_doc/qwen3_omni_moe.md, docs/source/en/model_doc/qwen3_next.md

Fast Path: Text Generation

The smallest useful text-generation test is a pipeline call. This confirms that Transformers can find the checkpoint, initialize the model, place it on available hardware, run tokenization, and decode output. Use device_map="auto" for large models when you want Transformers and its backend integrations to choose a placement strategy across available devices. The Qwen3.5 examples import torch, create the pipeline, and ask for a small number of new tokens so the smoke test completes quickly.

from transformers import pipeline
 
pipe = pipeline(
    task="text-generation",
    model="Qwen/Qwen3.5-9B",
    device_map="auto",
)
print(pipe("The capital of France is", max_new_tokens=20)[0]["generated_text"])

Once the smoke test works, switch to direct model loading when you need to inspect or transform inputs. This is the pattern used by the dense Qwen3.5, Qwen3.5 MoE, and Qwen3MoE pages. It makes each phase explicit: load tokenizer, load model, tokenize prompt, move tensors, call generate(), and decode the result. The same shape applies whether the class is Qwen3_5ForCausalLM, Qwen3_5MoeForCausalLM, or AutoModelForCausalLM; the difference is whether you bind to a concrete model family or let the auto class select one from the checkpoint configuration.

from transformers import AutoModelForCausalLM, AutoTokenizer
 
model_id = "Qwen/Qwen3-30B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
)
 
inputs = tokenizer("The key to effective reasoning is", return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Chat, Audio, and Multimodal Inference

Chat checkpoints often expect a structured conversation, not only a raw string. The Qwen3-Next example builds a messages list with role and content, then calls tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True). That produces the model-specific serialized prompt before tokenization. After generation, the example slices away the original prompt tokens and decodes only the newly generated IDs. This distinction matters for applications because the full generated tensor includes both prompt and continuation, while users typically want only the assistant response.

Sources: docs/source/en/model_doc/qwen3_next.md

from transformers import AutoModelForCausalLM, AutoTokenizer
 
model_name = "Qwen/Qwen3-Next-80B-A3B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
 
messages = [{"role": "user", "content": "Give me a short introduction to large language model."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
 
generated_ids = model.generate(**model_inputs, max_new_tokens=512)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
print(tokenizer.decode(output_ids, skip_special_tokens=True))

Speech and multimodal models follow the same loading concept but use processors to prepare richer inputs. In the Qwen3 ASR quickstart, AutoProcessor.from_pretrained(model_id) loads the preprocessing rules, AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto") loads the model, and processor.apply_transcription_request(audio=...) formats the request. The output is generated token IDs, just like text generation, but the raw decoded result may include language tags or task markers. The example therefore decodes the generated continuation after excluding input prompt IDs.

Model-Specific Runtime Notes

Quickstarts are intentionally short, but the model pages include runtime details that should influence your first production experiment. Qwen3.5 and Qwen3.5 MoE use hybrid linear-attention and full-attention layers; their docs note optional fast-kernel dependencies such as causal_conv1d and fla, with slower PyTorch fallbacks when those packages or GPU builds are unavailable. Qwen3.5 MoE also notes that fine-tuning should set output_router_logits=True so router logits are returned and load-balancing loss can be included, which is important for sparse expert stability.

Sources: docs/source/en/model_doc/qwen3_5.md, docs/source/en/model_doc/qwen3_5_moe.md

Model families are not interchangeable just because names look similar. The Qwen3.5 MoE page explains that the 35B-A3B text configuration has different shapes from dense Qwen3.5 checkpoints, with many experts and a smaller active path per token. Qwen3MoE likewise describes a mixture-of-experts architecture with total and active parameter counts that differ sharply. In practice, this means you should load the checkpoint with the matching model class or an auto class, avoid manually mixing dense and MoE weights, and read the model card when adapting code between related checkpoints.

Qwen3-Omni-MOE adds another class-selection concern. Its notes distinguish Qwen3OmniMoeForConditionalGeneration for audio and text output, Qwen3OmniMoeThinkerForConditionalGeneration for text-only output, and Qwen3OmniMoeTalkerForConditionalGeneration for audio-only output. They also call out that audio generation with the combined conditional generation class currently supports only single batch size, and that high-resolution video inputs can cause out-of-memory errors unless processor.max_pixels is decreased. These constraints should be checked before scaling a notebook example into a service.

Sources: docs/source/en/model_doc/qwen3_omni_moe.md

System-to-Code Mapping

Next Steps

After this page, use the Pipelines guide if you want task-level APIs for common inference jobs, or Auto Classes and Model Loading if you want to understand how AutoTokenizer, AutoProcessor, AutoModelForCausalLM, and model-specific classes are selected from checkpoints. Move to Text Generation for decoding parameters, chat templates, streamers, and KV-cache behavior. For audio, video, and multimodal inputs, read the processor pages before optimizing performance, because the processor defines the exact tensor structure the model expects.