Glossary
Purpose and Scope
The glossary is the documentation vocabulary layer for Transformers. Its job is to define general machine learning terms and library-specific terms so readers can understand tutorials, task guides, model reference pages, and examples without stopping to infer every phrase from context. The English source states this purpose directly, and the Arabic, Spanish, Japanese, and Korean glossary files preserve the same introductory intent for localized documentation. Read it as a concept map for the docs: it explains names that appear repeatedly in code, model class names, tokenizer outputs, preprocessing guides, and training objectives.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
The page is especially useful because many Transformers concepts are contracts between components rather than isolated dictionary definitions. An attention mask, for example, is produced by a tokenizer, travels inside a batch dictionary, and is consumed by a model so padding does not change the meaning of shorter sequences. A backbone is the feature-producing network body, while a head turns those features into a task prediction. These definitions help developers recognize why a model name, tokenizer output key, or preprocessing argument matters in a runnable workflow.
Sources: docs/source/en/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Relevant Source Files
- docs/source/en/glossary.md - canonical English glossary page, including the page purpose, doc-builder note, attention mask example, and early entries for model architecture, language modeling, and modality terms.
- docs/source/ar/glossary.md - Arabic localization that mirrors the glossary purpose and early entries, including the attention mask batching example and translated explanations for backbone, causal language modeling, channel, CTC, and convolution.
- docs/source/es/glossary.md - Spanish localization that keeps English technical anchors such as attention mask, autoencoding models, autoregressive models, and backbone while translating the reader-facing explanation.
- docs/source/ja/glossary.md - Japanese localization with the same attention mask and architecture entries, plus visible continuation into decoder input IDs and decoder model terminology.
- docs/source/ko/glossary.md - Korean localization with the same early glossary structure and visible continuation into DataParallel, showing that training and parallelism vocabulary also belongs in the glossary surface.
Core Vocabulary in the Batching Flow
The attention mask entry gives a concrete inference and training example rather than only a short definition. It loads a BertTokenizer with BertTokenizer.from_pretrained("google-bert/bert-base-cased"), tokenizes two English sequences, and shows that the encoded input_ids have different lengths. Because tensors in a batch need compatible shapes, the shorter sequence must be padded or the longer sequence must be truncated. The example chooses padding and passes both strings to the tokenizer with padding=True. That single argument changes the returned batch so both examples can be represented together.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
The important contract is that padding is represented in two related outputs. The input_ids list receives padding indices, shown as zeros added to the right side of the shorter sequence in the source example. The returned dictionary also contains attention_mask, a binary tensor-like list where 1 marks tokens the model should attend to and 0 marks padded positions. This makes batching safe: models can process equal-shaped tensors without treating artificial padding as real text. When debugging unexpected sequence behavior, inspect both input_ids and attention_mask together.
Sources: docs/source/en/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Model Architecture and Objective Terms
The glossary separates model architecture terms from task objective terms. A backbone is defined as the network body, including embeddings and layers, that outputs raw hidden states or features. It is usually connected to a head, which consumes those features to make a task-specific prediction. The English glossary uses ViTModel as an example of a backbone without a specific head on top, and it notes that other models such as DPT can use ViTModel as a backbone. This vocabulary explains why Transformers exposes base model classes and task-headed variants side by side.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Autoencoding models and autoregressive models are presented as cross-reference entries rather than long standalone definitions. Autoencoding models point toward encoder models and masked language modeling, while autoregressive models point toward causal language modeling and decoder models. That organization reflects how model families are commonly understood in Transformers: architecture, attention direction, and training objective reinforce one another. Causal language modeling is then defined as a pretraining task where a model reads text in order and predicts the next word, usually by masking future tokens at a given timestep.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Modality and Sequence Terms
The glossary also prepares readers for non-text tasks. The channel entry explains that color images combine red, green, and blue values, while grayscale images have one channel. It then names two tensor layouts that appear in Transformers image preprocessing: channel-first as n_channels, height, width, and channel-last as height, width, n_channels. This is more than visual terminology; it tells developers what shape conventions to look for when moving between image processors, model inputs, and framework tensors.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Speech and vision vocabulary appear beside the text terms because the library spans multiple modalities. Connectionist temporal classification, or CTC, is defined as an algorithm that lets a model learn without knowing the exact alignment between inputs and outputs. The glossary calls out speech recognition as a common use case because spoken audio does not align perfectly with transcripts. Convolution is defined as a neural network layer operation using a smaller kernel or filter over an input matrix, and the localized pages connect it to convolutional neural networks used in computer vision.
Sources: docs/source/ar/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Compact Reference
| Term or API surface | Source-level contract shown in the glossary | Where it appears in the workflow |
|---|---|---|
BertTokenizer.from_pretrained("google-bert/bert-base-cased") | Loads the tokenizer used in the attention mask example. | Tokenization before batching. |
tokenizer(sequence)["input_ids"] | Returns encoded token IDs for a single sequence. | Used to compare sequence lengths before padding. |
tokenizer([sequence_a, sequence_b], padding=True) | Requests padding for a list of sequences. | Builds a batch with compatible sequence lengths. |
input_ids | Tokenizer output containing token IDs plus padding indices when padding is applied. | Model input tensor construction. |
attention_mask | Tokenizer output key containing 1 for attended tokens and 0 for padded positions. | Prevents the model from attending to padding. |
ViTModel | Example of a backbone without a specific task head attached. | Vision model architecture and feature extraction. |
| DPT | Example of a model that can use ViTModel as a backbone. | Model composition across vision architectures. |
n_channels, height, width | Channel-first image tensor layout named by the glossary. | Image and vision preprocessing. |
height, width, n_channels | Channel-last image tensor layout named by the glossary. | Image and vision preprocessing. |
decoder_input_ids | Encoder-decoder input IDs supplied to the decoder; the Japanese source notes that models such as BART and T5 often create them from labels. | Sequence-to-sequence training such as translation and summarization. |
Sources: docs/source/en/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Localization and Documentation Behavior
The five glossary files show that this is a maintained documentation surface, not a one-off English note. The localized pages keep the same conceptual order and preserve many English API names, dictionary keys, class names, and anchors. That is intentional for developer documentation: translated explanations help local-language readers, while stable code terms such as attention_mask, input_ids, BertTokenizer, and ViTModel remain recognizable in Python examples. When changing glossary content, preserve headings and anchors whenever possible so links from tutorials and model docs continue to resolve.
Sources: docs/source/en/glossary.md, docs/source/ar/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
The English, Spanish, Japanese, and Korean files include a note that the page is Markdown with doc-builder-specific syntax, and the attention mask entry embeds a YouTube component. That means glossary edits should be reviewed as documentation source, not only as generic Markdown. Component syntax, cross-reference links, and code examples are part of the rendered reader experience. A good glossary contribution should improve clarity without weakening the concrete examples that connect definitions to library behavior.
Sources: docs/source/en/glossary.md, docs/source/es/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Practical Reading Path
Use the glossary whenever a guide introduces a term that affects how code is written. If the question is why a batch has both input_ids and attention_mask, start with the attention mask entry and then continue to preprocessing and tokenizers. If the question is why one model class is a base model and another has a prediction layer, start with backbone and head terminology before reading model reference pages. If the question is about language model training, connect autoencoding, autoregressive, causal language modeling, masked language modeling, encoder models, and decoder models.
Sources: docs/source/en/glossary.md, docs/source/ja/glossary.md, docs/source/ko/glossary.md
Next, read Tokenizers, Preprocessing Inputs, Configurations and Model Outputs, Auto Classes and Model Loading, and Task Overview. Those pages apply this vocabulary to runnable APIs and broader workflows: tokenizers return masks and IDs, processors prepare tensors for different modalities, model classes combine backbones and heads, and task guides explain why objectives such as causal language modeling or CTC are chosen. Treat the glossary as the vocabulary index, then move to those pages for implementation details and end-to-end examples.