Kubernetes and Helm
Apache Airflow supports Kubernetes both as a place to run the Airflow control plane and as a task execution substrate. In this repository, the packaged Kubernetes deployment interface is the Helm chart under chart/, while the KubernetesExecutor language SDK test shows how pod-per-task execution behaves in a more advanced runtime scenario. Use this page to orient deployment choices before reading production, scheduler, executor, or Task SDK details. It explains what the checked-in chart promises, how chart-specific Docker images fit into the workflow, and how KubernetesExecutor can be validated with Python, Go, and Java task workloads.
Sources: chart/README.md, chart/dockerfiles/README.md, kubernetes-tests/lang_sdk/README.md
Purpose and Scope
The Helm chart exists to bootstrap an Airflow deployment on a Kubernetes cluster by using Helm as the package manager. That makes the chart more than a collection of example manifests: it is the repository’s Kubernetes packaging surface for installing Airflow services, choosing executors, connecting to supported databases, enabling monitoring features, and applying deployment-time conveniences such as database migration and administrator account creation. The chart README points readers to the published Helm chart documentation for the stable release and notes that development-branch chart documentation is generated separately, so operators should distinguish source-tree behavior from release-specific published instructions. Sources: chart/README.md
Kubernetes in Airflow should be understood as two related but separate decisions. First, an organization can deploy Airflow itself on Kubernetes, using the chart to run scheduler-facing and web-facing components, supporting services, and worker infrastructure. Second, it can choose an executor that changes how task instances run. The official KubernetesExecutor terminology says each task instance runs in its own pod on a Kubernetes cluster, and the repository chart lists KubernetesExecutor alongside LocalExecutor and CeleryExecutor. A Kubernetes-hosted Airflow deployment therefore does not automatically imply pod-per-task execution, but the chart supports that mode when selected. Sources: chart/README.md
This page focuses only on Kubernetes and Helm artifacts represented by the requested source paths. It does not attempt to reproduce the full chart parameter reference or every Kubernetes provider operator. Instead, it gives maintainers and operators a source-backed map of the deployment capabilities named in the chart README, the image-publishing workflow used for chart helper images, and the end-to-end test architecture that validates language SDK coordination on KubernetesExecutor. The practical outcome is a clearer separation between Helm values, executor choice, image customization, and workload-specific pod templates. Sources: chart/README.md, chart/dockerfiles/README.md, kubernetes-tests/lang_sdk/README.md
Relevant Source Files
chart/README.md— Defines the Helm chart purpose, requirements, supported Airflow and executor versions, database support, monitoring features, security enhancements, deployment conveniences, and documentation locations.chart/dockerfiles/README.md— Documents chart-specific Docker images, the per-imagebuild_and_push.shscript, theapache/airflowDockerHub publishing requirement, and theDOCKERHUB_USERoverride for testing personal image builds.kubernetes-tests/lang_sdk/README.md— Describes an end-to-end KubernetesExecutor system test for a DAG mixing Python, Go, and Java tasks through SDK coordinator routing, object storage, init containers, and per-queue pod templates.
Deployment Model and Requirements
The chart README states the baseline requirements as a Kubernetes 1.30 or newer cluster, Helm 3.0 or newer, and optional persistent volume provisioner support in the underlying infrastructure. Persistent volumes are phrased as optional because a short-lived test deployment can be built with embedded or temporary backing services, but durable production deployments commonly need persistent storage for logs, metadata-adjacent services, DAG distribution, or other stateful integrations. Treat the README requirements as the source-tree contract for main, and treat published chart documentation as the exact release-specific installation guide when installing a chart version from Artifact Hub. Sources: chart/README.md
Executor support is intentionally broad. The chart supports LocalExecutor, CeleryExecutor, and KubernetesExecutor for all Airflow versions named by the README, plus multiple executors. It also recognizes executor classes from providers: airflow.providers.amazon.aws.executors.batch.AwsBatchExecutor, airflow.providers.amazon.aws.executors.ecs.AwsEcsExecutor, and airflow.providers.edge3.executors.EdgeExecutor. This lets one Helm packaging surface cover simple local-style execution, distributed Celery worker execution, pod-per-task Kubernetes execution, and provider-backed runtime models. The choice affects scaling, isolation, permissions, object storage, queues, and how workload images or pod templates are prepared.
Sources: chart/README.md
Database support in the chart README is centered on PostgreSQL and MySQL, with PostgreSQL and PgBouncer called out as having a battle-tested configuration. Because the Airflow metadata database stores scheduling state, task instance state, serialized metadata, and many UI-visible records, database readiness is not a peripheral concern. The chart also lists automatic database migration after a new deployment and administrator account creation during deployment. Those conveniences simplify bootstrap, but production operators should still decide who owns database privileges, migration timing, rollback strategy, and connection-pooling observability before promoting a chart deployment to shared use.
Sources: chart/README.md
Helm Chart Capabilities Reference
The following table summarizes concrete names and source-level contracts visible in the requested repository evidence. It is intentionally compact: use it to identify the exact capability or option family, then move to the published Helm chart documentation for full values, defaults, and upgrade notes. Sources: chart/README.md, chart/dockerfiles/README.md, kubernetes-tests/lang_sdk/README.md
| Area | Concrete names from source | Source-level contract |
|---|---|---|
| Chart purpose | Helm chart for Apache Airflow | Bootstraps an Airflow deployment on a Kubernetes cluster using Helm. |
| Requirements | Kubernetes 1.30+, Helm 3.0+, optional PV provisioner | Defines the main-branch minimum environment described by the chart README. |
| Core executors | LocalExecutor, CeleryExecutor, KubernetesExecutor | Supported executor choices for chart deployments. |
| Multiple executors | Supported multiple Executors | Allows chart deployments to use Airflow multiple-executor support. |
| AWS executors | airflow.providers.amazon.aws.executors.batch.AwsBatchExecutor, airflow.providers.amazon.aws.executors.ecs.AwsEcsExecutor | Supported with AWS provider version 8.21.0+. |
| Edge executor | airflow.providers.edge3.executors.EdgeExecutor | Supported with edge3 provider version 1.0.0+. |
| Airflow version | 3.1+ | Supported Airflow version range named by the chart README. |
| Databases | PostgreSQL, MySQL, PgBouncer | Supported database backends, with PostgreSQL and PgBouncer highlighted as battle-tested. |
| Celery autoscaling | KEDA | Provides autoscaling for CeleryExecutor. |
| Monitoring | StatsD, Prometheus, PgBouncer metrics, Flower | Monitoring surfaces named as chart features. |
| Security | container-specific Service Account Token Volume configuration | Only scheduler containers receive Kubernetes API access; init and sidecar containers operate without tokens. |
| Chart image build | build_and_push.sh, DOCKERHUB_USER | Each chart image directory has a build-and-push script; contributors can push test images to a personal DockerHub namespace. |
| SDK coordinator test | [sdk] coordinators, extra.pod_template_file, KubernetesExecutor | Per-queue pod template routing is used to run mixed Python, Go, and Java tasks. |
Security, Monitoring, and Operational Defaults
Security is not presented as an afterthought in the chart README. The chart supports container-specific Service Account Token Volume configuration that implements the principle of least privilege. The documented model gives Kubernetes API access only to scheduler containers, while init and sidecar containers operate without service account tokens. The README further describes defense in depth using both ServiceAccount and pod-level controls, with compatibility for policy systems such as Kyverno and broader compliance frameworks. For operators, this means chart-based deployments should be reviewed at the container-role level rather than assuming every container in an Airflow pod needs identical Kubernetes permissions. Sources: chart/README.md
Monitoring support is also part of the packaged story. The chart README lists StatsD and Prometheus metrics for Airflow, Prometheus metrics for PgBouncer, and Flower. Those names imply several complementary operational views: Airflow service metrics, database connection-pool metrics, and worker inspection for Celery-style deployments. The README also calls out KEDA autoscaling for CeleryExecutor, so autoscaling and observability should be considered together. A production deployment should connect these emitted signals to an organization’s monitoring stack before relying on Kubernetes restart behavior alone for operational feedback. Sources: chart/README.md
The README’s “one-command deployment” feature is valuable for evaluation because it says users do not need to provide external services such as Redis or a database just to test Airflow. That convenience should not be mistaken for production guidance. A disposable cluster can validate that the chart installs and that the selected executor starts; a durable installation must revisit database ownership, broker choices, persistence, chart values, network policy, service account permissions, log retention, and upgrade procedures. The repository source points readers to full Helm chart documentation for those production-level details. Sources: chart/README.md
Chart Docker Images and Contributor Workflow
The chart has its own Dockerfile-related support for images needed by the Helm chart. The dockerfiles README says each image directory contains a build_and_push.sh script that builds and pushes that image. It also states that pushing to the official apache/airflow DockerHub registry requires PMC membership with direct push access. This separates trusted official image publication from local or contributor validation. A maintainer changing a chart helper image can build the image with the checked-in script but should not assume they can publish it under the official namespace.
Sources: chart/dockerfiles/README.md
For testing image changes, the README documents the DOCKERHUB_USER variable. Setting that variable allows a contributor to push to a personal DockerHub namespace to test an image or build a custom image. That detail matters during Kubernetes debugging because failures may originate from Helm values, Kubernetes permissions, executor selection, or the container image that actually runs in a pod. A disciplined workflow isolates those variables: publish a test image to a personal namespace, point a test deployment at it, verify behavior, and only then move toward an official image publishing path if the change belongs in the chart image set.
Sources: chart/dockerfiles/README.md
KubernetesExecutor and Language SDK Test Flow
The language SDK Kubernetes test documents an end-to-end scenario where one DAG named lang_sdk_combined mixes Python, Go, and Java tasks and runs successfully on KubernetesExecutor. The test uses per-queue extra.pod_template_file routing added to the [sdk] coordinators configuration. A stub DAG is uploaded to a dags bucket, localstack provides S3-compatible storage in the cluster, and the scheduler routes golang and java queues to different worker pod templates. This demonstrates KubernetesExecutor behavior beyond a minimal Python-only pod launch because it validates artifact distribution, queue routing, and coordinator startup together.
Sources: kubernetes-tests/lang_sdk/README.md
The key runtime constraint in the test is that coordinator mode does not run the Python task runner inside the worker pod. Because of that, the ordinary DAG-bundle path is not responsible for downloading the Go binary or Java jar. Instead, an init container runs stage_artifacts.py, which reuses the DagBundle interface through DagBundlesManager().get_bundle(name).initialize(). The README describes this as the download half of task_runner.parse, used to pull artifacts from S3 buckets into shared paths that the coordinator scans before forking the Go binary or Java jar.
Sources: kubernetes-tests/lang_sdk/README.md
The component table in the test README provides a practical architecture map. dags/lang_sdk_combined.py is the Python stub DAG, go_example/ and java_example/ build the non-Python artifacts, pod_templates/lang_sdk_golang.yaml and pod_templates/lang_sdk_java.yaml define queue-specific worker pods, and config/values.yaml supplies Helm overrides for KubernetesExecutor, coordinators, queue routing, a stub-DAG S3 bundle, an AWS connection, and the scheduler pod-template mount. The Go binary, Java jar, and stub DAG share localstack as one object store but live in separate buckets: go-artifacts, java-artifacts, and dags.
Sources: kubernetes-tests/lang_sdk/README.md
Next Steps
Start with the Helm chart quick start when the goal is to prove that Airflow can run in a Kubernetes cluster. Before production, review the chart’s stable documentation for configuration, resource settings, log management, KEDA autoscaling, service account token volume examples, and upgrade guidance. If task isolation is the main goal, compare KubernetesExecutor with CeleryExecutor, LocalExecutor, and multiple-executor options rather than assuming Kubernetes hosting automatically chooses pod-per-task execution. For advanced Task SDK workloads, study the language SDK Kubernetes test because it shows how DAG bundles, object storage, init containers, pod templates, queues, and coordinators work together.