Summarization and Translation

Purpose and Scope

Summarization and translation are grouped together because both are sequence-to-sequence tasks: a model receives an input sequence and generates a new output sequence. Translation converts text from one language to another, while summarization produces a shorter version of a longer document that preserves the important information. The translation task guide states this framing directly and names summarization as another task that fits the same pattern, so the practical workflow is shared: load a conditional generation checkpoint, tokenize inputs and targets, train with generated predictions, and evaluate generated text rather than only class labels.

Sources: docs/source/en/tasks/translation.md

The repository documentation presents translation as a task-oriented guide rather than a low-level API reference. It teaches a complete path using T5 on the English-French subset of OPUS Books, then uses the fine-tuned model for inference. That structure is a useful template for summarization as well: choose a dataset with source-target pairs, preprocess the source text and target text together, use a sequence-to-sequence data collator, compute a generation metric, and run generation at inference time. The task guide is localized across multiple languages, which signals that the workflow is intended as a stable reader-facing recipe rather than an English-only example.

Sources: docs/source/en/tasks/translation.md, docs/source/ar/tasks/translation.md, docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md, docs/source/zh/tasks/translation.md

Relevant Source Files

• docs/source/en/tasks/translation.md - English task guide defining translation as a sequence-to-sequence task and showing the OPUS Books, T5, tokenizer, preprocessing, data collator, evaluation, and inference flow.
• docs/source/ar/tasks/translation.md - Arabic localization of the same translation guide, preserving the task framing, setup commands, dataset loading, and preprocessing requirements.
• docs/source/ja/tasks/translation.md - Japanese localization that includes the dataset split, tokenizer setup, target handling, batched mapping, and dynamic padding guidance.
• docs/source/ko/tasks/translation.md - Korean localization that repeats the OPUS Books workflow and emphasizes separate source and target processing before training.
• docs/source/zh/tasks/translation.md - Chinese localization with the preprocessing and SacreBLEU evaluation sections visible in the supplied source evidence.
• docs/source/fr/in_translation.md - French documentation placeholder indicating that the localized French translation page is still in progress.

Core Primitives

The core primitives for these tasks are a checkpoint, tokenizer, source text, target text, data collator, metric, and generation call. A checkpoint such as google-t5/t5-small supplies the pretrained model and tokenizer vocabulary. Source text is the model input, such as English sentences for translation or articles for summarization. Target text is what the model should learn to generate, such as French translations or concise summaries. The tokenizer must process both sides of the pair, and the guide uses text_target to tell the tokenizer that the labels are target-language text rather than another input field.

Sources: docs/source/en/tasks/translation.md, docs/source/zh/tasks/translation.md

The translation guide also introduces task prompts for multitask models. T5 is trained to condition on a prefix, so the input is prefixed with text such as translate English to French: . Summarization uses the same idea with a summarization prefix when required by the selected checkpoint. This matters because the model class alone does not always encode the task; the prompt can be part of the contract between the dataset and the pretrained model. If results look unrelated to the task, confirm that the prefix matches the checkpoint family and the intended task.

Sources: docs/source/en/tasks/translation.md, docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md

Dynamic padding is another important primitive. The localized guides show DataCollatorForSeq2Seq, which builds batches for sequence-to-sequence training and pads each batch to the longest example in that batch. That is more efficient than padding the entire dataset to a single maximum length. In summarization, where documents can vary widely in length, dynamic padding can substantially reduce wasted computation. In translation, it helps maintain efficient batches when sentence lengths differ across examples and languages.

Sources: docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md, docs/source/zh/tasks/translation.md

Fine-tuning Workflow

Start by installing the task dependencies. The translation guide installs Transformers, Datasets, Evaluate, and SacreBLEU. For summarization, the same stack is used with a summarization metric such as ROUGE. The guide also encourages logging in with huggingface_hub.notebook_login() so the fine-tuned model can be uploaded and shared. Treat login as optional for local experiments, but useful for reproducibility because the resulting checkpoint, tokenizer files, and model card can be associated with a Hub repository.

Sources: docs/source/en/tasks/translation.md

pip install transformers datasets evaluate sacrebleu

For translation, the documented dataset is OPUS Books with the en-fr configuration. The examples load it with load_dataset("opus_books", "en-fr"), split the training set with train_test_split(test_size=0.2), and inspect an example containing a translation dictionary with en and fr keys. The summarization equivalent is a dataset with one column for the document and one column for the summary. The important abstraction is the same: each row must provide a source string and the target string the model should learn to generate.

Sources: docs/source/en/tasks/translation.md, docs/source/ar/tasks/translation.md, docs/source/zh/tasks/translation.md

from datasets import load_dataset
 
books = load_dataset("opus_books", "en-fr")
books = books["train"].train_test_split(test_size=0.2)

Preprocessing is where most sequence-to-sequence mistakes happen. The guide creates a preprocess_function that builds prompted inputs from examples["translation"], extracts French targets, and calls the tokenizer with text_target=targets, max_length=128, and truncation=True. That pattern should be adapted rather than copied blindly: change source_lang, target_lang, the prefix, and maximum lengths for your dataset. For summarization, maximum input length and target length often need separate consideration because articles are much longer than summaries.

Sources: docs/source/en/tasks/translation.md, docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md

from transformers import AutoTokenizer
 
checkpoint = "google-t5/t5-small"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
 
source_lang = "en"
target_lang = "fr"
prefix = "translate English to French: "
 
def preprocess_function(examples):
    inputs = [prefix + example[source_lang] for example in examples["translation"]]
    targets = [example[target_lang] for example in examples["translation"]]
    model_inputs = tokenizer(inputs, text_target=targets, max_length=128, truncation=True)
    return model_inputs

After defining preprocessing, map it across the dataset with batched=True. Batched mapping is highlighted in the localized guides because it processes multiple examples at once and is faster for full datasets. Then construct DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint). The collator is connected to the checkpoint so it can prepare labels and padding consistently with the model family. This step is shared by translation and summarization because both train a conditional generator to predict token sequences, not a single class index.

Sources: docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md, docs/source/zh/tasks/translation.md

Evaluation and Generation Settings

Evaluation for generation tasks should compare decoded text with references. The Chinese guide shows evaluate.load("sacrebleu") for translation and a postprocessing function that strips predictions and wraps each label in a list before metric computation. That wrapping matters because BLEU-style translation metrics can accept multiple references per prediction. Summarization typically uses ROUGE instead, but the same lifecycle applies: decode predictions, replace ignored label values if needed, strip whitespace, and compute the metric on text rather than raw token ids.

Sources: docs/source/zh/tasks/translation.md

Generation settings control the shape of the output. Although the translation snippets focus on preprocessing and metric setup, the task itself depends on generated sequences at inference time. In practice, review options such as maximum generated length, beam search, length penalties, and early stopping for the checkpoint and task. Translation outputs should preserve meaning in the target language, while summarization outputs should be concise without dropping key facts. If outputs are too short, too repetitive, or in the wrong language, inspect both preprocessing and generation configuration before changing the model.

Sources: docs/source/en/tasks/translation.md

The most important distinction from classification is that predict_with_generate-style evaluation or an explicit generation call is required to measure the text the model would actually emit. Token-level loss is useful during training, but users experience the generated translation or summary. For translation, test with a few source sentences that contain punctuation, names, and idioms. For summarization, test with articles longer than the target length and verify that the model is not copying the full input. Small qualitative checks often reveal prompt, truncation, or target-column bugs before aggregate metrics do.

Sources: docs/source/en/tasks/translation.md, docs/source/zh/tasks/translation.md

Multilingual Documentation Signals

The same translation guide appears in Arabic, Japanese, Korean, and Chinese, and the source snippets preserve the same high-level structure: open in Colab, introduce translation, install libraries, load OPUS Books, split the dataset, preprocess with a tokenizer, map the dataset, create a sequence-to-sequence data collator, and evaluate. This consistency is useful when maintaining task docs because code examples, API names, and conceptual sequencing should remain aligned across localizations even when explanatory text differs.

Sources: docs/source/ar/tasks/translation.md, docs/source/ja/tasks/translation.md, docs/source/ko/tasks/translation.md, docs/source/zh/tasks/translation.md

There are also small localization differences worth noticing. Some translations describe target handling as setting text_target; others explain it as tokenizing source and target separately. These are compatible explanations of the same requirement: labels for generated text must be prepared as target text, not as another source-only input. The French path is an in-progress placeholder, so it should not be treated as a complete task guide. When updating the English page, maintainers should expect downstream localization work to mirror code blocks and headings closely.

Sources: docs/source/fr/in_translation.md, docs/source/en/tasks/translation.md

Next Steps

Use this workflow when you have paired source and target text and want a model that generates a new sequence. For translation, begin with the OPUS Books recipe and change source_lang, target_lang, prefix, and dataset configuration for your language pair. For summarization, keep the same tokenizer, collator, training, and generation pattern, but use document-summary columns and a summarization metric. After a first training run, inspect decoded examples, tune maximum lengths and generation parameters, and only then scale to larger checkpoints or longer datasets.

Sources: docs/source/en/tasks/translation.md, docs/source/zh/tasks/translation.md