Task Logs
Purpose and Scope
Task logs are the operator-facing record of what happened while an Airflow task instance ran. In the Airflow UI, the task instance log view has to balance two different jobs: it must show large streams efficiently, and it must preserve enough structured detail for troubleshooting. The supplied UI tests describe that contract from the user’s point of view. They validate a virtualized log list, expandable log groups, a keyboard-controlled source display, structured log rendering, and the text generated for downloads. Together, these behaviors define how a reader should inspect task output, narrow noisy logs, and export the view for support or incident analysis. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/Logs.test.tsx, airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/logDownloadContent.test.ts, airflow-core/src/airflow/ui/src/components/renderStructuredLog.test.tsx
This page focuses on the browser-facing task log experience rather than worker-side log collection. The evidence comes from tests around the Task Instance Logs page and the shared structured-log renderer. That is still a useful operational boundary: when a task fails, most users begin by opening the task instance in the UI, expanding the relevant groups, checking timestamps and levels, and optionally downloading a filtered text copy. The test cases show which details are intentionally visible, which task identity fields are summarized elsewhere, and how filters affect both the on-screen rendering path and the exported download content.
Relevant Source Files
- airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/Logs.test.tsx — Exercises the Task Instance Logs page, including the virtualized list, group headers, collapsed groups, and keyboard toggling of logger and location details.
- airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/logDownloadContent.test.ts — Reconstructs the download text path by parsing streaming log content, rendering each structured line in text mode, filtering empty results, and joining the remaining lines.
- airflow-core/src/airflow/ui/src/components/renderStructuredLog.test.tsx — Validates the shared renderer for structured log lines, including task-instance context stripping, custom field rendering, and task identity preamble output.
UI Rendering Model
The log page is designed for long task output, so the tests wait for a virtualized list before interacting with rows. Virtualization means the page renders only a working set of visible log items rather than every log line at once. The tests account for both normal log rows and group headers, then scroll the list to force visible items to render. For users, this explains why log views can remain responsive even when a task produces many lines. For maintainers, it means assertions and UI changes should treat the list as a window over the log stream, not as a static block of text where every row is always mounted in the document. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/Logs.test.tsx
Grouped logs are another first-class part of the task log experience. The task log grouping test navigates to a task instance route and expects summary headers such as Log message source details and Pre task execution logs to be visible. The comments state that group headers follow a summary naming pattern and are always visible, while groups start collapsed. This supports a troubleshooting workflow where a user first scans major phases, opens only the section that looks relevant, and avoids losing context in a flat stream. Collapsed groups are especially useful for separating setup, source metadata, and actual task execution messages when a failure is buried in verbose output. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/Logs.test.tsx
Structured Log Fields and Source Details
Structured log records can contain both the message that humans read and attributes that help explain where the message came from. The shared renderer deliberately treats task-instance identity fields as context rather than ordinary per-line attributes. The tested context fields are ti_id, dag_id, task_id, run_id, try_number, and map_index. When a structured log line contains these fields, the renderer still displays the event text, but it does not repeat those task identity values beside every line. This keeps repetitive metadata from overwhelming the log stream while allowing the same values to be rendered in a separate task identity preamble when needed. Sources: airflow-core/src/airflow/ui/src/components/renderStructuredLog.test.tsx
Non-task-instance structured fields are handled differently. The renderer test passes a custom key and verifies that the custom field is still visible while the task instance identifier is suppressed. This distinction matters when operators add application-specific context to their logs: custom attributes remain useful as per-line troubleshooting hints, while standard Airflow task identity stays centralized. The same test file also validates a preamble helper in both text and JSX-oriented modes. In text mode, the preamble returns key and value pairs, optionally prefixed by a label such as Task Identity, and only includes fields that are present in the supplied context. Sources: airflow-core/src/airflow/ui/src/components/renderStructuredLog.test.tsx
Source visibility is intentionally controlled by the user. The log page test confirms that logger and location details are hidden by default, then become visible after the user sends the S keyboard shortcut. After the toggle, a row containing the DagBag message exposes the logger as source equals airflow.models.dagbag.DagBag and the location as dagbag.py line 593. That behavior gives a practical escalation path: start with clean, readable messages; if the error is ambiguous, enable source details to identify the logger name and file location that produced the line. This is particularly helpful when framework setup messages and task code messages are interleaved. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/Logs.test.tsx
Filtering, Search, and Download Semantics
The download path mirrors the structured rendering path rather than dumping raw response objects. The test constructs the same shape used by the page download logic: parse the streaming log response, render each line through the structured renderer in text mode, remove empty strings, and join the remaining lines with newline separators. The renderer receives log level filters, source filters, a log link value, timestamp display settings, and a source visibility flag. In the tested download construction, timestamps are shown, source details are hidden, and rendering mode is text. This makes the downloaded file a faithful text representation of the filtered view rather than an unprocessed API payload. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/logDownloadContent.test.ts
Level filtering has strict consequences for exported content. If every structured line is excluded by the selected level filter, the generated download text is empty. If a structured line lacks a level field and any log level filter is active, that line is also excluded. The tests also guard a subtle usability issue: when earlier lines are filtered out, the download must not begin with stray newline characters. A filtered file containing only the visible info line should contain that line, omit the hidden debug marker, and start directly with content. These cases prevent confusing blank exports and preserve clean artifacts for incident notes or support tickets. Sources: airflow-core/src/airflow/ui/src/pages/TaskInstance/Logs/logDownloadContent.test.ts
For practical search and investigation, treat filters, grouping, timestamps, and optional source display as complementary tools. Start by opening the task instance log, scan the visible group summaries, and expand the phase that corresponds to the failure. Use level filtering to reduce debug or informational noise before downloading a file, because the same exclusion behavior is applied to the download text. If the message points into Airflow internals or provider code, enable source details with the keyboard shortcut and capture the logger and location values. If structured custom fields appear, use them as application-level breadcrumbs while relying on the task identity preamble for the canonical DAG, task, run, try, and map context.
Implementation Reference
| Component | Source-backed contract |
|---|---|
| Virtualized list | Task log rows and group headers are rendered inside a virtualized list, and tests scroll it before checking row content. |
| Group headers | Summary headers are visible and groups start collapsed, supporting phase-oriented navigation through log output. |
| Source toggle | Logger and location fields are hidden by default and shown after the S keyboard shortcut. |
| Structured renderer | Task identity fields are stripped from per-line attributes, while non-identity custom fields render normally. |
| Text preamble | Task identity context can be rendered as key and value pairs, with an optional label and only present fields included. |
| Download content | Streaming log content is parsed, rendered in text mode, filtered for empty lines, and joined without leading newline artifacts. |
Operational Troubleshooting Flow
A dependable task-log workflow starts with context, then narrows toward the failing line. First confirm the task instance route, run, try, and map index through the surrounding UI context or task identity preamble. Next scan grouped sections, because pre-execution messages and source-detail sections can explain failures that occur before user task code runs. Then use level filters to isolate warnings or errors, remembering that lines without a level are excluded when a level filter is active. Finally, download the filtered text when you need to attach evidence to a ticket, compare retries, or preserve an incident timeline outside the browser.
The tests also reveal a useful boundary for authors of DAGs and operators. Logs become easier to diagnose when messages carry meaningful events and structured custom fields rather than only unstructured text. Airflow’s renderer will avoid repeating the standard task identity fields per line, so adding custom fields is a way to surface domain-specific details without competing with built-in DAG and task metadata. When logs are too noisy, group boundaries and filters help, but they cannot replace clear event names and consistent levels. For related topics, continue to Logging Architecture for configuration and extension points, Metrics, Traces, and Health Checks for deployment observability, and Callbacks and Error Handling for failure-time recovery hooks.