Connections

Purpose and Scope

A connection is Airflow's named description of how a task, hook, or provider integration reaches an external system. It usually contains a connection identifier, a connection type, host and port information, login credentials, schema or database name, and an extra JSON field for provider-specific options. The official Airflow connection guide describes several storage and management surfaces: environment variables, a secrets backend, the metadata database, the UI, the CLI, and export files. In day-to-day DAG authoring, the most important habit is to pass a stable connection id to operators and hooks, rather than embedding credentials in DAG files.

Connections sit at the boundary between reusable workflow code and deployment-specific configuration. A DAG can say that a task should use aws_default, postgres_reporting, or another logical name, while each Airflow environment decides where that connection is stored and which secret values it resolves to. This separation is also why provider packages matter: a provider can define its own connection type, fields, UI behavior, and hook mapping so the same generic Airflow connection mechanism can serve databases, SaaS APIs, object stores, message queues, and other systems. Sources: devel-common/src/sphinx_exts/providers_extensions.py

Relevant Source Files

• devel-common/src/sphinx_exts/providers_extensions.py - implements provider documentation extension utilities used to render provider-owned reference material, including the core extensions documentation area where provider connection metadata is summarized.
• providers/fab/src/airflow/providers/fab/www/extensions/init_appbuilder.py - initializes the Flask AppBuilder integration used by the FAB-based web UI, including app setup, security manager wiring, menu/view registration, and auth-manager integration.
• providers/fab/src/airflow/providers/fab/www/extensions/init_jinja_globals.py - adds runtime values to Jinja templates, including the active auth manager, configured UI colors, timezone, version information, plugin state, and optionally redacted hostname.
• providers/fab/src/airflow/providers/fab/www/extensions/__init__.py - marks the FAB web extension package that groups the UI initialization helpers used by Airflow's FAB provider.
• docs/images/documentation_architecture.py - generates a documentation architecture diagram showing that package documentation is published from the apache-airflow repository into the live Airflow documentation site.

User-Facing Connection Surfaces

Airflow supports more than one way to supply a connection because installations vary in security posture and operational workflow. The official guide documents environment-variable storage, JSON connection representations, URI connection strings, secrets backend lookup, metadata-database storage, UI creation and editing, CLI creation, connection export, and connection testing. These are not separate kinds of connection; they are different ways of materializing the same logical record. When a hook asks for a connection id, Airflow resolves that id through the configured storage mechanisms and returns the connection data needed by the provider hook.

The UI path is important because many teams delegate connection maintenance to operators rather than DAG authors. The FAB provider source shows how Airflow initializes Flask AppBuilder for the web application, including the framework class, session and security manager wiring, base templates, static paths, plugin enablement, view registration, and auth-manager integration. That UI infrastructure is broader than connections alone, but it is the source-backed web layer through which connection forms, provider-defined fields, and secure administrative workflows are presented in FAB-based deployments. Sources: providers/fab/src/airflow/providers/fab/www/extensions/init_appbuilder.py, providers/fab/src/airflow/providers/fab/www/extensions/init_jinja_globals.py

For scripted environments, prefer declarative and repeatable connection setup. The official docs show both JSON and URI formats, and they call out details such as arbitrary dictionaries in extra and special characters in connection parameters. JSON is usually clearer for complex extras because it avoids URI escaping mistakes; URI format is compact and convenient for environment variables. Either way, DAG code should keep using the connection id. That lets development, staging, and production supply different endpoints and credentials without editing or redeploying DAG files.

Provider Connection Metadata

Providers extend the connection model by declaring connection types and metadata for Airflow's documentation and UI. The provider connections reference is described as a summary of community-managed provider implementations, and it explains that each provider can define custom parameters, UI customizations, field behaviors, and connection types. Those connection types can then be used to automatically create provider hooks for specific external systems. This is the contract that lets Airflow core stay generic while provider packages contribute integration-specific knowledge.

The repository includes a Sphinx extension module for provider documentation that is loaded across provider docs. It imports provider package data, defines utilities for inspecting provider source with Python ast, and renders Jinja-based reference directives. Although the supplied snippet is infrastructure rather than a provider's provider.yaml, it grounds how provider-owned metadata becomes first-party documentation. For connection readers, the practical takeaway is that provider reference pages are generated from provider distributions and should be treated as the authoritative place to discover provider-specific connection types, hook names, and custom fields. Sources: devel-common/src/sphinx_exts/providers_extensions.py

A custom provider can expose connection metadata in provider.yaml, while legacy Python metadata remains documented for compatibility. In both cases, the goal is the same: make the connection form understandable and make the connection type resolvable by hooks. Custom fields should be named for the external service's real concepts, but should still fit Airflow's connection shape. Use extra for structured options that do not belong in the common host, login, password, schema, or port fields, and document required versus optional values in the provider reference.

Secure Handling and UI Context

Connections are security-sensitive because they often contain credentials, tokens, hostnames, account identifiers, and environment topology. The official guide distinguishes secrets backend storage from database storage and includes a section on the security of database-stored connections. In production, prefer a secrets backend when credentials should be managed outside Airflow's metadata database. If database storage is used, treat Airflow administrative access, backups, exports, and logs as sensitive paths because they may reveal or indirectly expose connection information.

The FAB Jinja initialization code demonstrates the same general security posture in the UI layer: runtime context is deliberately prepared before being exposed to templates, and the hostname is only included when the fab.EXPOSE_HOSTNAME configuration allows it; otherwise it is set to a redacted value. The context also injects the active auth manager, which allows templates to work against the configured authorization layer. While that code is not a connection serializer, it shows the surrounding UI design principle: expose only the operational context the web interface needs, and route authorization through Airflow's configured manager. Sources: providers/fab/src/airflow/providers/fab/www/extensions/init_jinja_globals.py

Connection testing should be used as a validation tool, not as a substitute for least privilege. A successful test tells you that Airflow can resolve the connection and that the provider's hook can perform its test behavior, but it does not prove that every task operation has the correct permission set. Keep credentials scoped to the smallest role that supports the DAG, avoid sharing one high-privilege connection across unrelated workflows, and rotate secrets in the backing secret store rather than editing DAG code.

System-to-Code Mapping

Concern	Source-backed implementation signal	Reader impact
Provider-generated connection reference	devel-common/src/sphinx_exts/providers_extensions.py loads provider package data and renders provider documentation extensions.	Look up provider-specific connection types and fields in provider reference docs.
FAB web application shell	providers/fab/src/airflow/providers/fab/www/extensions/init_appbuilder.py initializes the AppBuilder-based UI framework.	Connection forms and admin flows run inside this secured web UI layer.
Template runtime context	providers/fab/src/airflow/providers/fab/www/extensions/init_jinja_globals.py registers Jinja globals such as auth manager, timezone, UI colors, versions, plugin state, and redacted hostname.	UI pages receive controlled operational context rather than arbitrary environment data.
Extension package boundary	providers/fab/src/airflow/providers/fab/www/extensions/__init__.py identifies the FAB extension package.	UI setup helpers are grouped under the FAB provider extension namespace.
Documentation publishing	docs/images/documentation_architecture.py models package docs being published from the apache-airflow repository to the live docs site.	Generated provider connection documentation is part of the published Airflow documentation system.

Practical Workflow

Start by choosing the provider and hook or operator that will talk to the external system. Read that provider's connection documentation to identify the connection type and any required custom fields. Next, create a connection id that describes the purpose rather than the credential owner, such as warehouse_readonly or s3_data_lake. Store the actual values using the mechanism appropriate for the environment: an environment variable for simple local use, a secrets backend for production, or the metadata database when UI-managed administration is acceptable.

When encoding a connection, prefer JSON for complex options and reserve URI strings for simple, script-friendly cases. Put provider-specific structured values in extra, and verify special characters are escaped correctly if URI format is used. After creation, test the connection using the supported UI or CLI workflow, then reference only the connection id in DAGs, operators, hooks, and task code. This keeps DAGs portable and allows platform teams to rotate credentials, change endpoints, or move storage backends without changing workflow source.

Next Steps

If you are selecting an integration, continue with providers-overview-and-installation and operators-and-hooks-reference to find the provider package, hook, and operator that match the target system. If you are extending Airflow, read custom-providers and core-extension-contracts before defining new connection metadata. If your immediate concern is credential protection, pair this page with secrets-backends-and-masking and security-model so connection storage, masking, UI access, and deployment trust boundaries are designed together.