Training Scripts

Purpose and Scope

Training scripts are the maintained, runnable examples that show how to fine-tune Transformers models for real tasks without starting from a blank training loop. The English training-script guide frames them as examples for PyTorch tasks in the repository examples area, while the Arabic and German versions broaden the same idea to PyTorch, TensorFlow, and JAX/Flax example families. The important contract is that these scripts are intentionally readable examples, not universal applications. You should expect to run them first on a small slice of data, then edit their preprocessing, dataset fields, model choice, and training arguments for your own task.

Sources: docs/source/en/run_scripts.md, docs/source/ar/run_scripts.md, docs/source/de/run_scripts.md

The project distinguishes actively maintained examples from research-project and legacy scripts. The run-script docs warn that research and legacy examples are not actively maintained and may require a specific Transformers version that is not compatible with the latest library. That distinction matters when debugging: a current example script should be paired with a source install and its local requirements, while an older recipe should be checked out from the matching release tag. If you are copying older code, treat the tag as part of the reproducibility environment, not as an optional detail.

Sources: docs/source/en/run_scripts.md, docs/source/de/run_scripts.md, docs/source/ar/run_scripts.md

Relevant Source Files

• docs/source/en/run_scripts.md — primary English guide for installing from source, running a summarization example, truncating datasets for smoke tests, resuming checkpoints, and pushing trained models to the Hub.
• docs/source/en/mixed_precision_training.md — explains the TrainingArguments precision switches that are commonly passed to example scripts when adapting them for faster or lower-memory training.
• docs/source/de/run_scripts.md — German localization of the run-script guide, useful confirmation that the same maintained-versus-legacy distinction and setup sequence are part of the public documentation contract.
• docs/source/de/training.md — German fine-tuning tutorial that connects example scripts to the broader fine-tuning workflow: dataset preparation, tokenization, Trainer, TensorFlow Keras, and native PyTorch.
• docs/source/ar/run_scripts.md — Arabic localization of the run-script guide, including the same expectations around transparent preprocessing and contribution discussions before adding script features.
• docs/source/ar/training.md — Arabic fine-tuning tutorial that grounds the data-preparation and framework-choice concepts used when adapting training scripts.

Script Organization and Maintenance Model

The docs organize training examples by framework and task rather than by model family alone. A reader is pointed toward PyTorch examples, TensorFlow examples, and JAX/Flax examples, and the English run-script guide uses summarization as the concrete walkthrough. The localized run-script guides also describe summarization examples for PyTorch and TensorFlow and say examples should work with both frameworks unless stated otherwise. In practice, this means you should choose the script by task first, then verify that the framework-specific folder supports the model architecture and command-line options you intend to use.

Sources: docs/source/en/run_scripts.md, docs/source/de/run_scripts.md, docs/source/ar/run_scripts.md

Readability is an explicit design constraint for example scripts. The docs say most scripts fully expose how data is preprocessed so that users can edit it for their use case, and they caution contributors to discuss new features on the forum or in an issue before opening a pull request. Bug fixes are welcome, but new functionality that makes examples harder to read is unlikely to be merged. When adapting a script internally, follow the same principle: prefer a small, understandable change to the dataset mapping or argument defaults over turning the example into a private framework.

Sources: docs/source/en/run_scripts.md, docs/source/ar/run_scripts.md, docs/source/de/run_scripts.md

Setup and Execution Flow

Start from a clean virtual environment and install Transformers from source when running the latest example scripts. The run-script docs repeat this requirement across languages because examples on the main branch can depend on unreleased library behavior. After installing the library, move into the example folder you selected and install that example’s requirements.txt. If you intentionally need an older script, check out the matching tag before installing requirements so the script, dependencies, and library API line up with each other.

Sources: docs/source/en/run_scripts.md, docs/source/de/run_scripts.md, docs/source/ar/run_scripts.md

git clone https://github.com/huggingface/transformers
cd transformers
pip install .
git checkout tags/v3.5.1
pip install -r requirements.txt

The recommended first run is a smoke test on truncated data. The English guide calls out max_train_samples, max_eval_samples, and max_predict_samples as parameters for limiting work before committing to a full run that may take hours. It also warns that not all scripts support max_predict_samples, so use the script’s help output to inspect the available arguments. For summarization, the documented example fine-tunes T5-small on CNN/DailyMail and uses an additional source_prefix because T5 is prompted to summarize.

Sources: docs/source/en/run_scripts.md

python examples/pytorch/summarization/run_summarization.py -h

Checkpointing and publishing are part of the normal script lifecycle. The run-script guide describes --resume_from_checkpoint path_to_specific_checkpoint for continuing interrupted training from a checkpoint folder. It also describes --push_to_hub, which creates a Hub repository and uploads the model to the folder name from --output_dir, and --push_to_hub_model_id, which lets you choose the repository name explicitly. Treat these as operational controls: they do not change the model objective, but they do affect recoverability and artifact sharing.

Sources: docs/source/en/run_scripts.md

Adapting Scripts to Custom Data

Adapting a script usually starts in preprocessing. The fine-tuning tutorials describe loading a dataset, inspecting examples, creating a tokenizer with AutoTokenizer.from_pretrained, and using the 🤗 Datasets map method to apply a preprocessing function across the dataset. The shown text-classification tutorial uses padding and truncation to normalize variable-length text sequences, then optionally creates smaller shuffled subsets for quicker iteration. Those same ideas apply to example scripts: find the function that maps raw examples into model inputs, then align it with your dataset’s column names and task labels.

Sources: docs/source/de/training.md, docs/source/ar/training.md

The framework-specific training path depends on how much control you need. The fine-tuning docs present three supported approaches: Trainer, TensorFlow with Keras, and native PyTorch. Example scripts commonly choose a high-level path so users get argument parsing, evaluation, checkpointing, and saving behavior without writing the whole loop. If your customization is only data formatting, labels, metrics, or training arguments, stay close to the script. If you need to change the forward pass, loss computation, or distributed strategy beyond exposed arguments, use the script as a reference and move the custom code into a clearer project-specific training entry point.

Sources: docs/source/de/training.md, docs/source/ar/training.md

Training Options Reference

Use the following options as the compact command surface documented by the supplied guides. Availability is script-specific, so always confirm with the script’s -h output before adding an option to automation.

Sources: docs/source/en/run_scripts.md, docs/source/en/mixed_precision_training.md

Mixed precision is often the first performance adaptation after a script works functionally. The mixed-precision guide defines full precision as fp32 training and mixed precision as using fp16 or bf16 for compute-intensive forward and backward passes while keeping an fp32 master copy for optimizer updates. It warns to load the model in fp32 when using autocast-based mixed precision; otherwise autocast becomes a no-op and the optimizer lacks the fp32 master weights expected by that workflow. Choose bf16 on newer Ampere-class hardware when possible, and fp16 on older GPUs.

Sources: docs/source/en/mixed_precision_training.md

Next Steps

A practical workflow is to select the maintained example for your task, install from source, install that example’s requirements, run a small-sample smoke test, and then edit the preprocessing block for your dataset. Once the script trains and evaluates correctly, add operational flags for checkpoint resume, Hub upload, and precision. If you need deeper customization, read the fine-tuning and Trainer pages next so you can decide whether to extend the existing script, subclass the training behavior, or write a clearer task-specific entry point that reuses the same Transformers primitives.

Sources: docs/source/en/run_scripts.md, docs/source/de/training.md, docs/source/ar/training.md