Chat Templates

Chat templates are the formatting layer between a user-facing conversation and the token sequence consumed by a causal language model. A chat history is convenient to store as a list of messages, where each message has a role such as user, assistant, or system and a content payload. The model, however, does not receive that structured list directly. It receives tokens. A template defines how roles, message boundaries, generation prompts, end-of-message markers, and special control tokens are serialized before tokenization and generation.

Sources: docs/source/ar/chat_templating.md

The key practical rule is that chat-trained models are still causal language models continuing a sequence. Fine-tuning teaches a model to respond to a particular serialized conversation format, and different checkpoints can use different formats even when they descend from the same base model. Transformers stores the expected format on the tokenizer or processor so application code can pass message dictionaries rather than hand-writing prompt strings for each checkpoint. This is why apply_chat_template is the preferred entry point for chat formatting.

Sources: docs/source/ar/chat_templating.md

Purpose and Scope

Use this page when you need to understand what a chat template does, how to pass messages to it, when to use tokenizer versus processor templating, and how to write or test templates safely. It focuses on the developer workflow around message formatting: create a structured conversation, let the checkpoint-specific template serialize it, optionally tokenize the result, and pass the prepared inputs to generation or a chat-capable pipeline. The same idea applies to plain text chats and multimodal chats, but multimodal content uses richer content blocks instead of a single string.

Sources: docs/source/ar/chat_templating.md

This page intentionally treats templates as a public model-contract concern rather than a cosmetic prompt helper. A model may use tags such as [INST], role headers such as <|user|>, an end-of-sequence token, or a model-specific assistant prefix. Small formatting differences can change model behavior because the model was trained to condition on those markers. The safest application design is to keep conversations as structured data as long as possible and call the library template method at the boundary where text or tensors are needed.

Sources: docs/source/ar/chat_templating.md

Relevant Source Files

• docs/source/ar/chat_templating.md - Localized repository documentation for chat templates. It explains that templates belong to the tokenizer, convert message lists into the model-specific string format, and show concrete AutoTokenizer.apply_chat_template examples for BlenderBot, Mistral Instruct, and Zephyr.
• docs/source/en/testing.md - English model-testing guidance. It identifies CausalLMModelTest, VLMModelTest, and ALMModelTest, and shows that generation and pipeline behavior are covered by shared test mixins for language and multimodal model families.
• docs/source/ro/testing.md - Romanian version of the same model-testing guidance. It reinforces the shared distinction between causal language models, vision-language models, audio-language models, and the multimodal parent test pattern.
• docs/source/de/testing.md - German testing guide for repository-level CI behavior and common pytest commands. It documents fast pull-request tests, slow scheduled tests, and the tests and examples suites used to validate behavior.
• docs/source/ja/testing.md - Japanese testing guide that mirrors repository testing flows, including RUN_SLOW=1 pytest tests/ and RUN_SLOW=1 pytest examples/ for broader validation.
• docs/source/ko/testing.md - Korean testing guide that documents test selection, CI categories, and pytest patterns for running targeted or keyword-matched checks.
• docs/source/en/trainer_recipes.md - Trainer feature documentation used here only for training-time context: it shows that model outputs expose logits and that causal language-model training uses shifted labels, which is relevant when chat-formatted text becomes training examples.

Core Primitives

A text-only chat template is attached to a tokenizer, usually loaded through AutoTokenizer.from_pretrained(checkpoint). The tokenizer owns special tokens, vocabulary, and model-specific serialization rules, so it is the natural place for chat_template and apply_chat_template. With tokenize=False, the method returns the formatted string, which is useful for inspection and debugging. With tokenization enabled, it produces token ids ready to feed into a language model or the generation stack.

Sources: docs/source/ar/chat_templating.md

A multimodal chat template is handled by a processor rather than only a tokenizer. A processor is the preprocessing object that can combine text tokenization with image, audio, video, or other modality preparation. In multimodal conversations, content is not just a string; it is a list of typed blocks, for example a text block and an image block in the same user message. The processor template serializes the textual conversation structure while also preparing raw modality inputs needed by the model.

Sources: docs/source/en/testing.md, docs/source/ro/testing.md

A generation prompt is the template-added prefix that tells a chat model it is now the assistant’s turn to continue. In common usage, set add_generation_prompt=True when formatting a prompt for generate() or for a lower-level inference call. This adds the assistant header or equivalent marker without adding assistant content. Do not confuse it with a system prompt: the system prompt is a message in the conversation, while the generation prompt is a structural marker at the end of the serialized context.

Sources: docs/source/ar/chat_templating.md

Text Chat Formatting Flow

The normal text chat flow starts by loading a tokenizer for the checkpoint, building a list of message dictionaries, and calling apply_chat_template. Each dictionary carries the role and the content. For example, a conversation can contain a user greeting, an assistant reply, and a second user request. The template then emits the exact serialized format expected by that checkpoint. A simple template may mostly join turns with whitespace, while an instruction-tuned model may insert instruction delimiters, role tags, and end markers.

Sources: docs/source/ar/chat_templating.md

from transformers import AutoTokenizer
 
checkpoint = "mistralai/Mistral-7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
 
messages = [
    {"role": "user", "content": "Hello, how are you?"},
    {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
    {"role": "user", "content": "Show me how chat templating works."},
]
 
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
 )
print(prompt)

Inspecting the formatted prompt is a useful debugging step when adopting a new checkpoint. It lets you confirm whether the model expects bracketed instruction blocks, explicit role headers, separator tokens, or an end-of-sequence marker after each assistant turn. Once the string looks correct, switch to tokenized outputs for generation. Keeping the same structured messages object also makes it easier to swap checkpoints because the checkpoint’s tokenizer supplies the formatting contract.

Sources: docs/source/ar/chat_templating.md

Writing and Saving Templates

A custom chat template is a Jinja template stored in the tokenizer’s chat_template attribute. Jinja gives template authors Python-like control flow for iterating over messages, checking roles, printing message content, and conditionally appending the assistant generation prompt. The template should be written as the model’s training format, not as a generic transcript format. If the model was fine-tuned with end-of-message markers, assistant headers, or role-specific wrappers, the template should reproduce those markers exactly.

Sources: docs/source/ar/chat_templating.md

{%- for message in messages %}
{{- '<|' + message['role'] + '|>\n' }}
{{- message['content'] + eos_token }}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|assistant|>\n' }}
{%- endif %}

After assigning a template to tokenizer.chat_template, the same template is used by apply_chat_template. The official workflow also treats the template as part of the tokenizer artifact: saving or pushing the tokenizer persists the template so downstream users do not need to rediscover the prompt format. A practical authoring loop is to print an existing model’s template, simplify it for the model family you are adapting, test the formatted strings with representative conversations, and only then save or publish the tokenizer.

Sources: docs/source/ar/chat_templating.md

Multimodal Chat Templates

Multimodal chat models extend the same conversation abstraction to inputs such as images, audio, or video. Instead of content being a single string, each message’s content is a list of typed items. A user turn can therefore include an image item followed by a text item asking a question about the image. The processor, not just the tokenizer, is responsible for aligning that structured chat with the model’s text tokens and raw modality features.

Sources: docs/source/en/testing.md, docs/source/ro/testing.md

messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "You are a concise visual assistant."}],
    },
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "http://images.cocodataset.org/val2017/000000039769.jpg"},
            {"type": "text", "text": "What are these?"},
        ],
    },
]

The repository testing documentation reflects this modality split in its model-test base classes. CausalLMModelTest covers causal language models, while VLMModelTest and ALMModelTest cover vision-language and audio-language models. The multimodal test parent places modality placeholder tokens in input_ids alongside raw audio or vision features. That testing structure mirrors the runtime contract for multimodal chat templates: text tokens preserve the conversation skeleton, while modality features are carried in parallel inputs prepared by the processor.

Sources: docs/source/en/testing.md, docs/source/ro/testing.md

System-to-Code Mapping

The mapping matters because most bugs appear at boundaries. If a prompt string is hand-built, it can silently omit a role tag or add a duplicate end token. If a multimodal message is treated as plain text, the processor does not receive the image, audio, or video block it needs. If a model family is added without generation or pipeline tests, template regressions can escape local validation. Keep each concern in its owning object: tokenizer for text chat formatting, processor for multimodal preprocessing, and model tests for generated behavior.

Sources: docs/source/ar/chat_templating.md, docs/source/en/testing.md, docs/source/ro/testing.md

Testing Signals

Chat-template changes should be validated at two levels: local formatting checks and model-family behavior checks. For local checks, run targeted tests or inspect formatted prompts for a few representative conversations. For model behavior, the repository’s model-testing documentation recommends task-specific files such as tests/models/mymodel/test_modeling_mymodel.py, exact unittest selectors, keyword filters, and RUN_SLOW=1 for slow integration coverage. This is especially important for templates that affect generation, pipelines, or multimodal placeholder handling.

Sources: docs/source/en/testing.md, docs/source/de/testing.md, docs/source/ja/testing.md, docs/source/ko/testing.md

# run one model test file
pytest tests/models/mymodel/test_modeling_mymodel.py -v
 
# run one test method
pytest tests/models/mymodel/test_modeling_mymodel.py::MyModelTest::test_model
 
# run integration tests for a model package
pytest tests/models/mymodel/ -k integration -v
 
# include slow integration tests
RUN_SLOW=1 pytest tests/models/mymodel/ -v

The CI documentation in the localized testing guides also distinguishes fast pull-request checks from scheduled slow tests over tests and examples. That split is useful when deciding how much evidence a template change needs. A small tokenizer-template correction may be covered by focused tests and prompt inspection. A new multimodal chat format, processor contract, or generation behavior change should receive broader generation and pipeline validation because the serialized chat affects the full path from preprocessing through logits and decoded output.

Sources: docs/source/de/testing.md, docs/source/ja/testing.md, docs/source/ko/testing.md, docs/source/en/trainer_recipes.md

Next Steps

When using an existing chat model, start with AutoTokenizer or the model’s processor and call apply_chat_template instead of writing a prompt string by hand. Inspect the formatted output once with tokenize=False, then use tokenized outputs or a chat-capable pipeline for inference. When publishing or adapting a checkpoint, treat the template as part of the tokenizer or processor artifact and test representative conversations before pushing. For deeper follow-up, read the pages on Text Generation, Pipelines, Processors, Tokenizers, and Tool Use and Response Parsing.

Sources: docs/source/ar/chat_templating.md