Troubleshooting and Debugging

Purpose and Scope

This page helps you decide which Transformers troubleshooting path to use when a script fails during setup, model loading, training, or model development. The project’s published troubleshooting guide is intentionally practical rather than exhaustive: it points readers to the Hugging Face forums for questions, GitHub issues for likely library bugs, and the migration guide when behavior changed across versions. That framing matters because many failures are environmental or data-dependent rather than defects in Transformers itself. A good debugging report should include the failing command, versions, hardware, stack trace, and minimal reproducible code before escalating to maintainers.

Sources: docs/source/en/troubleshooting.md, docs/source/ar/troubleshooting.md, docs/source/ja/troubleshooting.md, docs/source/ko/troubleshooting.md

The repository separates common user troubleshooting from deeper debugging tools. The troubleshooting page covers recognizable error messages such as cache download failures in firewalled environments, CUDA out-of-memory errors, ImportError for recently released model classes, TensorFlow save/load mismatches in localized pages, and CUDA device-side assertions. The debugging page targets distributed training and numerical failures, especially underflow and overflow detection. The internal model debugging page serves model adders who need structured forward-call traces while porting or validating a new architecture. Treat these as escalating layers: quick configuration fixes first, numerical instrumentation second, model-porting trace capture last.

Sources: docs/source/en/troubleshooting.md, docs/source/en/debugging.md, docs/source/en/internal/model_debugging_utils.md

Relevant Source Files

• docs/source/en/troubleshooting.md — English user-facing troubleshooting guide for common runtime, installation, model-loading, and CUDA errors.
• docs/source/ar/troubleshooting.md — Arabic localization of the same troubleshooting flow, including common fixes and help channels.
• docs/source/ja/troubleshooting.md — Japanese localization; includes the TensorFlow saved-model guidance and CUDA troubleshooting sections visible in the source evidence.
• docs/source/ko/troubleshooting.md — Korean localization; mirrors the common troubleshooting categories and preserves localized anchors for reader navigation.
• docs/source/en/debugging.md — Debugging guide for distributed training categories and DebugUnderflowOverflow usage through TrainingArguments.debug or a manual PyTorch loop.
• docs/source/en/internal/model_debugging_utils.md — Internal toolbox documentation for model_addition_debugger_context, its intended audience, output files, and example workflow.

Common Troubleshooting Flow

Start by classifying the failure before changing code. If the script stalls while downloading a model or dataset and then reports that requested files cannot be found in the cached path, the likely cause is a firewalled cloud or intranet environment. The troubleshooting guide recommends running Transformers in offline mode for that case, which changes the problem from network access to cache preparation. In practice, that means pre-populating the cache from a connected environment or ensuring the required checkpoint files already exist locally before launching training or inference on the restricted machine.

Sources: docs/source/en/troubleshooting.md

When the failure is memory-related, the guide names CUDA out-of-memory as a common training problem for large models. The first fixes are deliberately simple: reduce per_device_train_batch_size in TrainingArguments, or use gradient_accumulation_steps so the effective batch size can remain larger while each device processes fewer examples at once. This guidance is useful because it keeps the model, optimizer, and dataset constant while reducing peak activation memory. If the job still fails, continue with the performance and optimization docs, but record the original batch size, sequence length, precision mode, and GPU capacity.

Sources: docs/source/en/troubleshooting.md, docs/source/ko/troubleshooting.md

For import and model-loading errors, check version and serialization assumptions before rewriting the application. The troubleshooting guide gives ImportError: cannot import name 'ImageGPTImageProcessor' from 'transformers' as an example that can occur when code expects a newly released class but the installed package is older. The recommended command is a package upgrade. The localized troubleshooting pages also describe TensorFlow saved-model confusion: model.save stores more TensorFlow-specific state than Transformers necessarily reloads, so the safer Transformers workflow is save_pretrained and from_pretrained, or saving weights as tf_model.h5 and reloading from the containing folder.

Sources: docs/source/en/troubleshooting.md, docs/source/ja/troubleshooting.md, docs/source/ar/troubleshooting.md

pip install transformers --upgrade

from transformers import TFPreTrainedModel
 
model.save_pretrained("path_to/model")
model = TFPreTrainedModel.from_pretrained("path_to/model")

Debugging Numerical and Distributed Training Failures

The debugging guide defines four broad categories for distributed training problems: numerical issues, communication failures, runtime errors, and build errors. The most concrete tool described in the source evidence is underflow and overflow detection. Underflow and overflow are numerical failures where activations, weights, or loss values become inf, nan, or loss=NaN. Instead of guessing which layer first produced invalid values, enable DebugUnderflowOverflow so forward hooks inspect module inputs, outputs, and corresponding weights after forward calls. This is especially relevant for mixed precision training, where large or tiny values can appear early.

Sources: docs/source/en/debugging.md

There are two supported entry patterns. If you use Trainer, set debug="underflow_overflow" in TrainingArguments. If you have a custom PyTorch training loop, import DebugUnderflowOverflow from transformers.debug_utils and attach it to the model yourself. The emitted report is not just a yes-or-no signal. It includes the batch number where the invalid value was detected and a frame-by-frame view of recent forward calls, with absolute minimum and maximum values for weights, inputs, and outputs. Read it from the bottom upward to identify the first suspicious module before the final inf or nan.

Sources: docs/source/en/debugging.md

from transformers import TrainingArguments
 
args = TrainingArguments(
    debug="underflow_overflow",
    ...
)

from transformers.debug_utils import DebugUnderflowOverflow
 
debug_overflow = DebugUnderflowOverflow(model)

Model-Addition Debugging Toolbox

model_addition_debugger_context is a power-user tool for contributors adding or porting models into Transformers. The internal documentation describes it as a context manager that tracks forward calls inside a model forward pass and writes nested JSON containing slices and summaries of inputs and outputs. It also enforces torch.no_grad(), so it is meant for inspection rather than training. Use it when you need to compare a new Transformers implementation against another implementation, inspect dtype or shape differences, or reduce manual tensor save/load work during a model port.

Sources: docs/source/en/internal/model_debugging_utils.md

The documented workflow uses a concrete multimodal example with LlavaProcessor and LlavaForConditionalGeneration. The important operational detail is that the context manager wraps model.forward(**inputs), not .generate(). That distinction keeps the trace focused on the model’s forward path instead of generation control flow. The context manager accepts a debug_path for output placement and do_prune_layers; setting do_prune_layers=False outputs all layers. After the forward call, the debugger writes two files with the same base name: one ending in _SUMMARY.json and one ending in _FULL_TENSORS.json.

Sources: docs/source/en/internal/model_debugging_utils.md

from transformers.model_debugging_utils import model_addition_debugger_context
 
with model_addition_debugger_context(
    model,
    debug_path="optional_path_to_your_directory",
    do_prune_layers=False,
 ):
    output = model.forward(**inputs)

The summary file is the first artifact to inspect because it records module paths, argument and keyword-input structure, tensor shapes, dtypes, and statistics such as mean, standard deviation, minimum, and maximum for tensors. The full-tensor artifact is heavier and should be used when summaries reveal a likely mismatch that requires exact values. This toolbox is narrower than the general troubleshooting page: it assumes you can run the model, build representative inputs, and reason about module paths. For routine user errors, prefer the public troubleshooting guide and numerical debugging hooks before generating full model traces.

Sources: docs/source/en/internal/model_debugging_utils.md

Quick Reference

Sources: docs/source/en/troubleshooting.md, docs/source/en/debugging.md, docs/source/en/internal/model_debugging_utils.md

Next Steps

When debugging, make the smallest change that tests one hypothesis. Confirm network and cache assumptions before investigating model code, reduce batch-level memory pressure before changing architecture, and upgrade the package before assuming a missing class is a source bug. For numerical failures, capture the first invalid forward frame rather than the final stack trace only. For model additions, save the summary JSON from model_addition_debugger_context alongside the reproduction so reviewers can inspect shapes, dtypes, and module paths. Related pages to read next are Installation for offline mode, Trainer for TrainingArguments, LLM optimization for memory pressure, and Contributor Guide for model-addition expectations.

Sources: docs/source/en/troubleshooting.md, docs/source/en/debugging.md, docs/source/en/internal/model_debugging_utils.md