Text Classification

Purpose and Scope

Text classification is the Transformers task for assigning one label or class to a piece of text. In the official guide, the representative example is sentiment analysis: a model reads a movie review and predicts whether it is negative or positive. This page explains the practical workflow behind that guide: install the supporting libraries, load a labeled dataset, tokenize text, build dynamically padded batches, evaluate with a metric, fine-tune a sequence classification model, and run inference with the result. It is intended for readers who want to adapt the IMDb example to their own labeled text datasets.

Sources: docs/source/en/tasks/sequence_classification.md, docs/source/ar/tasks/sequence_classification.md, docs/source/ko/tasks/sequence_classification.md, docs/source/pt/tasks/sequence_classification.md, docs/source/zh/tasks/sequence_classification.md

The sequence classification guide is localized across several documentation trees, but the core task remains the same in each version: fine-tune DistilBERT on stanfordnlp/imdb and then use the fine-tuned model for prediction. The multilingual pages also preserve the same reader-facing primitives, including load_dataset, AutoTokenizer, Dataset.map, DataCollatorWithPadding, Hugging Face Hub login, and a task-page pointer for compatible checkpoints. Treat those pieces as the canonical workflow for binary and multi-class sequence classification, and then change the dataset, label names, metric, or checkpoint for your own task.

This page focuses on sequence-level labels: the model produces one prediction for each text example or text pair. Multiple choice classification is a related text classification problem, but it uses a different example shape because each question is paired with several candidate endings or answers. Token classification is also related, but it predicts a label for each token rather than one label for the whole sequence; the Arabic token classification guide demonstrates that different alignment problem with WNUT 17, ner_tags, and word-piece tokenization. Use this page when the label belongs to the whole input text.

Sources: docs/source/ar/tasks/token_classification.md

Relevant Source Files

• docs/source/en/tasks/sequence_classification.md — Canonical English task guide for fine-tuning DistilBERT on IMDb and using the resulting model for inference.
• docs/source/ar/tasks/sequence_classification.md — Arabic localization of the same sequence classification workflow, including installation, Hub login, IMDb loading, preprocessing, and inference intent.
• docs/source/ko/tasks/sequence_classification.md — Korean localization that shows the same dataset fields, tokenizer setup, preprocess_function, Dataset.map, and dynamic padding pattern.
• docs/source/pt/tasks/sequence_classification.md — Portuguese localization that reinforces the task definition, IMDb example, label meanings, and DistilBERT tokenizer preprocessing.
• docs/source/zh/tasks/sequence_classification.md — Chinese localization with explicit dynamic padding guidance using DataCollatorWithPadding and the batched=True dataset mapping flow.
• docs/source/ar/tasks/token_classification.md — Related task guide used here only to distinguish sequence-level text classification from token-level labeling such as NER.

Core Primitives

A text classification run has four essential data primitives. The dataset must expose a text column and a label column; in the IMDb guide those fields are text and label, where 0 means a negative review and 1 means a positive review. The tokenizer converts text into model-ready input IDs and attention masks. The data collator assembles examples into batches and pads them to a common length. The metric converts model predictions and labels into an interpretable score, typically accuracy for a simple binary classification tutorial.

The model primitive is a checkpoint loaded with a sequence classification head. The guide uses distilbert/distilbert-base-uncased, a compact BERT-family encoder, and fine-tunes it so the classification head maps the pooled representation of each review to the IMDb labels. In your own project, the same pattern works for sentiment, topic, intent, toxicity, review rating, language identification, and similar tasks as long as each training example has one target label. For text pairs, such as premise and hypothesis, the tokenizer can receive both fields, but the output is still one label per pair.

The workflow also includes ecosystem primitives outside transformers. datasets.load_dataset retrieves and structures the training data, evaluate supplies metrics, accelerate supports the training stack used by Trainer, and huggingface_hub.notebook_login lets you authenticate before uploading a fine-tuned model. The official guide recommends installing these together because the tutorial crosses library boundaries: data comes from Datasets, the model and tokenizer come from Transformers, metrics come from Evaluate, and sharing goes through the Hub.

Sources: docs/source/en/tasks/sequence_classification.md, docs/source/zh/tasks/sequence_classification.md

pip install transformers datasets evaluate accelerate

>>> from huggingface_hub import notebook_login
>>> notebook_login()

Execution Flow

Start by loading the labeled dataset. The IMDb example uses load_dataset("stanfordnlp/imdb") and inspects a test record to show the expected schema. That schema is intentionally simple: raw review text and an integer class label. When adapting the guide, confirm the same two ideas exist in your data even if the column names differ. If your labels are strings, normalize them to stable IDs and keep an ID-to-label mapping so training, evaluation, and inference reports remain understandable after the model is saved.

>>> from datasets import load_dataset
>>> imdb = load_dataset("stanfordnlp/imdb")
>>> imdb["test"][0]

Next, load the tokenizer for the checkpoint you plan to fine-tune. The guide uses AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased"), which selects the tokenizer class associated with the checkpoint metadata. The preprocessing function calls the tokenizer on the dataset text field and enables truncation so inputs do not exceed DistilBERT's maximum length. Applying that function with Dataset.map(..., batched=True) processes multiple examples at once and is the standard way to transform a full Datasets split before training.

>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")
>>> def preprocess_function(examples):
...     return tokenizer(examples["text"], truncation=True)
tokenized_imdb = imdb.map(preprocess_function, batched=True)

Batching is handled with DataCollatorWithPadding. Instead of padding every example in the dataset to a global maximum length, the collator pads each batch to the longest sequence in that batch. The Chinese and Korean task pages call this out because dynamic padding is usually more efficient for training and evaluation. This detail matters when reviews vary greatly in length: it reduces wasted tokens while still producing rectangular tensors that the model can process on CPU, GPU, or accelerator-backed training loops.

>>> from transformers import DataCollatorWithPadding
>>> data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

After preprocessing, configure evaluation and training. For the IMDb tutorial, accuracy is a natural metric because there are two balanced labels and each example has exactly one correct class. A typical compute_metrics function converts logits to predicted label IDs, compares them with labels, and returns the metric dictionary expected by Trainer. Then TrainingArguments defines output paths, evaluation cadence, learning behavior, and optional Hub upload, while Trainer binds together the model, arguments, datasets, tokenizer or processing component, collator, and metric callback.

Sources: docs/source/en/tasks/sequence_classification.md, docs/source/ko/tasks/sequence_classification.md, docs/source/zh/tasks/sequence_classification.md

API Components and Compact Reference

Use the high-level APIs when you want a maintained training loop rather than a custom PyTorch loop. AutoTokenizer.from_pretrained(checkpoint) resolves the correct tokenizer implementation for the checkpoint. AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=...) loads the base model with a classification head sized for your labels. TrainingArguments records training configuration, and Trainer executes training, evaluation, saving, and prediction. For quick inference after training, you can either call the model and tokenizer directly or create a pipeline("text-classification", model=...) from the saved checkpoint.

The most important adaptation points are the checkpoint, dataset columns, and label configuration. If you switch from IMDb to a three-way sentiment dataset, set num_labels=3 and provide meaningful label mappings. If your text column is not named text, update the preprocessing function. If your examples contain two text fields, pass both to the tokenizer in a fixed order. Keep the collator and metric callback aligned with the tokenizer output and label format, because mismatched column names or label shapes are the most common source of training failures in this workflow.

Inference and Task Boundaries

Inference is the second half of the official sequence classification guide: after fine-tuning, use the trained model to classify new text. The simplest path is a text classification pipeline pointed at the saved local directory or Hub repository. The lower-level path is to tokenize a string, run the model, apply an argmax over logits, and translate the predicted ID back to a label name. The pipeline is better for quick application code, while the lower-level path is better when you need custom batching, device placement, calibration, or access to raw scores.

When your problem looks similar but the label granularity changes, choose the task carefully. Token classification, illustrated by the Arabic NER guide, starts from token sequences and ner_tags, uses is_split_into_words=True, and must align word-level entity labels with subword tokens. That is not the same as sequence classification, where the entire review or sentence gets one sentiment or topic label. Multiple choice classification is also not a drop-in replacement because each example contains a group of candidates. The shared ideas are tokenization, labels, metrics, and fine-tuning; the model head and data shape differ.

Sources: docs/source/ar/tasks/token_classification.md, docs/source/en/tasks/sequence_classification.md

Testing Signals and Next Steps

A healthy fine-tuning run should show that preprocessing preserved the expected labels, dynamic padding creates valid batch tensors, evaluation returns the metric keys you configured, and inference labels match your id2label mapping. Before training for a long time, inspect one raw example, one tokenized example, and one collated batch. Then run a tiny training or evaluation pass to catch column-name mistakes. After training, test a few positive and negative examples manually before uploading the model so the Hub artifact reflects the intended task and label semantics.

For a broader task map, continue to task-overview. For lower-level loading details, read auto-classes-and-model-loading. For reusable training-loop behavior, read trainer and trainer-api. If your labels belong to individual words or entities instead of the whole text, switch to token-classification. If your objective is generative, such as summarization or translation, use the generation-oriented task guides instead of adapting sequence classification beyond its natural shape.