Cloud Composer pricing

This document explains Cloud Composer pricing.

• To see the pricing for other products, read the Pricing documentation.

• To search for individual SKUs associated with Cloud Composer, see Google Cloud SKUs.

• You can use the Google Cloud Pricing Calculator to get a cost estimate for Google Cloud products, including Cloud Composer 2 and Cloud Composer 1.

Pricing models

Cloud Composer uses three pricing models:

• Cloud Composer 3 pricing
• Cloud Composer 2 pricing
• Cloud Composer 1 pricing

Cloud Composer environments are billed in short time intervals, for the actual time period when it was running. For example, if you create an environment, run it for 6 hours and 30 minutes, and delete it afterwards, then the total costs are for 6.5 hours.

Although some pricing is stated in hours or by the month, Cloud Composer is still billed for the actual usage time.

Cloud Composer 3 pricing

This section describes pricing in Cloud Composer 3.

Pricing table for Cloud Composer 3

This section summarizes Cloud Composer 3 costs for different regions. Also see additional costs.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Cloud Composer 3 SKUs

Cloud Composer 3 has the following SKUs:

• Cloud Composer 3 standard milli DCU-hours

  Data Compute Unit is an abstract metering unit that represents computational resources allocated by a Cloud Composer environment at a point in time. Because DCU is a measure of the currently provisioned resources of an environment, costs are charged for the environment's DCU value over time, DCU-hours.

  This SKU is measured in milliDCU-hours. For example, if your environment uses 1 DCU for 1 hour, this is equal to using 1000 milliDCU-hours.

  DCU-hours represent the usage of the following resources:

  • vCPU, Memory and Storage resources, used by the environment's components that run the workloads: Airflow workers, schedulers, DAG processors, triggerers, and the web server. Airflow workers are autoscaled, and as such the corresponding costs follow the changing number of workers in the environment.

    If you deploy your own workloads in your environment's cluster, such as when you create Pods using KubernetesPodOperator or execute tasks with Kubernetes Executor, then used resources also count as DCU-hours and follow Cloud Composer 3 pricing model.

  • Size of the environment. The environment size is the scale of the managed infrastructure of your Cloud Composer environment. Environment infrastructure comes in different sizes: Small, Medium, and Large.

    The cost of environment's size in DCUs covers the cost of resources used by infrastructure components required to run Airflow. For example, by the Cloud SQL instance (with a separate SKU covering the costs of storage) or by the Redis queue of your environment.

• Cloud Composer 3 database storage

  This SKU covers the cost of the storage used by the Cloud SQL instance that stores the Airflow database of your environment.

  The minimum disk size of the Cloud SQL instance is 10 GiB.

  The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage and does not decrease if the size of the Airflow database is reduced later.

• Cloud Composer Network Data Transfer

  This group of SKUs covers the billable outbound traffic generated by the environment's components that run the workloads in your environment.

  Cloud Composer network data transfer SKUs cover the following types of traffic generated by workloads runnng in your environment:

  • Intra-zone and Inter-zone data transfer
  • Inter-region data transfer
  • Egress to Google services: Data transfer to a different Google Cloud service within the same region using an external IP address or an internal IP address.
  • Internet data transfer rates with Premium Tier pricing.

  For more information about network egress billing and descriptions of the related SKUs, see Network Pricing.

• Your environment also has additional costs that are not a part of Cloud Composer pricing.

Additional costs for Cloud Composer 3

Costs for the following services are billed in addition to costs for Cloud Composer 3 environments:

• The Cloud Storage bucket of an environment, which is used for managing DAGs and holding task logs. This bucket persists unless manually deleted. Cloud Composer interacts with the bucket through Cloud Storage API, which generates additional operation charges. The additional costs are:

  • Cloud Storage data storage
  • Cloud Storage operation charges

• Creating and storing environment snapshots produces additional costs related to Cloud Storage. The costs depend on the snapshot creation frequency and the size of a snapshot. Snapshot size depends on the Airflow database size and the size of the data that is included in the snapshot. The additional costs are:

  • Cloud Storage operation charges
  • Cloud Storage outbound data transfer
  • Cloud Storage data storage

• The data that Cloud Monitoring collects to help you understand your environment's performance and health. The data is subject to separate Monitoring pricing.

• If you use Customer Managed Encryption Keys, there might be additional costs for the usage of the Cloud Key Management Service. See Cloud Key Management Service pricing for details.

Pricing example for Cloud Composer 3

For example, you create a Cloud Composer 3 environment in Iowa (us-central1), use it for 10 days to run your workloads, and then delete it. The environment operates during the a month that has 30 days.

As an example, your environment uses the following amount of resources:

• For 5 days it uses 12 DCUs. These DCUs cover the combined costs of vCPU, memory, and storage resources used by the environment's components ( schedulers, DAG processors, triggeres, workers, the web server), and the costs of operating the environment's infrastructure.
• For other 5 days your environment uses 15 DCUs because the number of workers was scaled up to address the large amount of tasks.
• Your environment's database does not go over the initial size of 10 GiB.
• The environment does not generate any billable network egress traffic because it operates within the same zone and does not send data over the internet. In this case all Cloud Composer Network Data Transfer SKUs do not generate costs.

In this case, the breakdown of costs is the following:

SKU	First period	Second period	Cost in Iowa (us-central1)	Total cost (10 days)
DCU-hours (or 1000 milliDCU-hours)	1440 DCU-hours (12 DCUs for 120 hours)	1800 DCU-hours (15 DCUs for 120 hours)	$0.06	$194.4 (3240 DCU-hours)
Database storage	1.67 GiBy/Mo (10 GiBy for 5 days out of 30 days)	1.67 GiBy/Mo (10 GiBy for 5 days out of 30 days)	$0.17	$0.5678 (10 GiBy for 10 days)

Additional considerations:

• Your environment is auto-scaling. This means that if the actual load is lower during the described period, then the costs are also lower. For example, if you configure your environment to scale between 1 and 6 workers and your environment uses only a single worker during the whole period, then you pay for DCU-hours used by 1 worker, not by 6 workers.

• If your environment transfers data between regions or over the internet, then Cloud Composer Network Data Transfer SKUs apply.

• Your environment also has additional costs that are not a part of Cloud Composer 3 SKUs.

Pricing model transition for Cloud Composer 3

This section describes how the pricing model changes in Cloud Composer 3 compared to Cloud Composer 2.

Cloud Composer 3 follows the path of further simplification and hides most of the remaining infrastructure components including the cluster of your environment and dependencies on other services such as Artifact Registry, Cloud Build, or Pub/Sub.

The overall costs of an Cloud Composer environment are now combined into a fewer number of SKUs:

• SKUs that represented Compute Engine resources used by your environment's components are combined into the DCU-hours SKU.

• SKUs for the environment's size are included into the DCU-hours SKU.

• Cloud Composer 3 Network Egress SKUs replace corresponding Cloud Composer 2 additional costs related to network traffic usage by the Airflow components that previously ran in your project.

• You do not pay additional costs for Google services that were utilized in Cloud Composer 2 by the environment's cluster located in your project.

Cloud Composer 2 pricing

This section describes pricing in Cloud Composer 2.

Pricing table for Cloud Composer 2

The following table summarizes Cloud Composer 2 costs for different regions. Also see Additional costs.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Cloud Composer 2 SKUs

Cloud Composer 2 has the following SKUs:

• Cloud Composer Compute SKUs

  Cloud Composer Compute SKUs represent Compute Engine capacity used by Airflow schedulers, web server and workers. Airflow workers are autoscaled, and as such the corresponding costs follow the changing number of workers in the environment.

  In addition, if you deploy your own workloads in your environment's cluster, then the pricing for these workloads also follows the Cloud Composer 2 pricing model and uses Compute Engine Compute SKUs. For example, pods started in your environment's cluster as Spot Pods follow Cloud Composer 2 Compute pricing model and not GKE Autopilot Spot pricing model.

  • Cloud Composer Compute CPUs

    Associated costs depend on the combined number of vCPUs used by all your environment's components that run on Compute Engine. This includes costs for pods and services in your environment's cluster. For example, all your environment's Airflow workers run in pods in your environment cluster.

    This SKU is measured in 1000 mCPU (millicores) per hour. For example, if your environment uses 1 vCPU for 1 hour, this is equal to using 1000 mCPU for 1 hour.

  • Cloud Composer Compute Memory

    Associated costs depend on the combined amount of memory used by all your environment's components that run on Compute Engine.

  • Cloud Composer Compute Storage

    Associated costs depend on the combined amount of storage used by all your environment's components that run on Compute Engine.

• Cloud Composer Database Storage

  Associated costs depend on the amount of disk space used by the Cloud SQL instance. The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage.

  This SKU component covers the cost of Airflow database storage.

  The minimum disk size of Cloud SQL instances is 10 GiB.

• Small/Medium/Large Cloud Composer Environment Fee

  Associated costs depend on the size of your environment. The environment size is the scale of the managed infrastructure of your Cloud Composer environment.

  This SKU covers the cost of infrastructure components required to run Airflow, including Cloud SQL database, task queue, connection proxies. Environment infrastructure comes in three different sizes: Small, Medium and Large.

• Small/Medium/Large Highly Resilient Cloud Composer Environment Fee

  These SKU represents the cost of infrastructure components required to run Airflow in a highly resilient Cloud Composer environment.

  If your environment is highly resilient, these SKUs replace the Small/Medium/Large Cloud Composer Environment Fee SKUs.

• Your environment also has additional costs that are not a part of Cloud Composer pricing.

Additional costs for Cloud Composer 2

Costs for the following services are billed in addition to costs for Cloud Composer 2 environments:

• The Cloud Storage bucket of an environment, which is used for managing DAGs and holding task logs. This bucket persists unless manually deleted. Cloud Composer interacts with the bucket through Cloud Storage API, which generates additional operation charges. The additional costs are:

  • Cloud Storage data storage
  • Cloud Storage operation charges

• Creating and storing environment snapshots produces additional costs related to Cloud Storage. The costs depend on the snapshot creation frequency and the size of a snapshot. Snapshot size depends on the Airflow database size and the size of the data that is included in the snapshot. The additional costs are:

  • Cloud Storage operation charges
  • Cloud Storage outbound data transfer
  • Cloud Storage data storage

• The data that Cloud Monitoring collects to help you understand your environment's performance and health. The data is subject to separate Monitoring pricing.

• The storage and outbound data transfer generated when using Artifact Registry. This service stores and serves Cloud Composer environment images. For more information, see Artifact Registry pricing pages.

• If you use Customer Managed Encryption Keys, there might be additional costs for the usage of the Cloud Key Management Service. See Cloud Key Management Service pricing for details.

• If you use Private Service Connect then the following additional charges are applied:

  • Networking Private Service Connect Consumer End Point, with the use of the Using a Private Service Connect endpoint (forwarding rule) to access a published managed service rule.
  • Networking Private Service Connect Consumer Data Processing.

• Billable outbound traffic generated by the environment's components. For more information about network egress billing and descriptions of the related SKUs, see Network Pricing.

Pricing example for Cloud Composer 2

Assume that you create a Cloud Composer 2 environment in Iowa (us-central1) and use the default Small environment preset.

In this case, your environment has the following default parameters that affect Cloud Composer 2 SKUs:

• Your environment has one scheduler.
• Your environment scales automatically between 1 and 3 workers.
• Your environment's scheduler and web server use 0.5 vCPU each.
• Your environment's scheduler and web server use 1.875 GiB of memory each.
• Your environment's scheduler and web server use 1 GiB of disk space each.
• Your environment's workers scale automatically between 0.5 and 1.5 vCPUs, depending on the number of workers.
• Your environment's workers scale automatically between 1.875 and 5.625 GiB of memory, depending on the number of workers.
• Your environment's workers scale automatically between 1 and 3 GiB of storage, depending on the number of workers.
• Your environment's database uses 10 GiB of storage. The storage size increases automatically, following the demand coming from the database storage usage. This example assumes that the database storage does not increase.
• Your environment uses the small infrastructure size.

Assume that you run this environment for 7 days and 12 hours (180 hours total). Your environment's load is 1 worker for 50% of the time and 2 workers for the other 50% of the time. After using the environment for this period of time, you delete it. In this case, your Cloud Composer 2 SKUs are:

• Cloud Composer Compute CPUs is ( 90 hours * 1.5 vCPU + 90 hours * 2 vCPU ) * $0.045 per 1000 mCPU hours, for a total of $14.175.

• Cloud Composer Compute Memory is ( 90 hours * 5.625 GiB + 90 hours * 7.5 GiB ) * $0.005 per GiB / hour, for a total of $5.906.

• Cloud Composer Compute Storage is ( 90 hours * 3 GiB + 90 hours * 4 GiB ) * $0.0002 per GiB / hour, for a total of $0.126.

• Cloud Composer Database Storage is 180 hours out of 740 hours * 10 GiB * $0.17 per GiB / month, for a total of $0.413.

• Small Cloud Composer Environment Fee is 180 hours * $0.35 per hour, for a total of $63.00.

• The total Cloud Composer 2 fees in this example are:

  Cloud Composer 2 SKU	Fee
  Cloud Composer Compute CPUs	$14.175
  Cloud Composer Compute Memory	$5.906
  Cloud Composer Compute Storage	$0.126
  Cloud Composer Database Storage	$0.17 per GiB / month
  Small Cloud Composer Environment Fee	$63.00
  Total	$83.62

• Your environment is auto-scaling. This means that if the actual load is lower during the described period, then the costs are also lower. For example, if you configure your environment to scale between 1 and 6 workers, and your environment uses only a single worker during the whole period, you only pay for this single worker.

• Your environment also has additional costs that are not a part of Cloud Composer 2 SKUs.

Pricing model transition for Cloud Composer 2

This section describes how the pricing model changes in Cloud Composer 2 compared to Cloud Composer 1.

Introduction of Cloud Composer 2 features is accompanied by a new pricing model, enabling you to fully benefit from the efficiency of autoscaling. The new model also provides a clear perspective on a Total Cost of Ownership for Cloud Composer environments.

In Cloud Composer 1 environments, the cost of the Compute Engine layer used to run Airflow is visible as a charge for Compute Engine instances used by environment's GKE cluster.

In Cloud Composer 2 this cost is no longer associated with Compute Engine. Instead, it is included in Cloud Composer 2 SKUs for Compute Engine CPU cores, Memory and Storage. This approach provides a clear perspective on the overall cost of Cloud Composer environments.

Because the pricing model of Cloud Composer 2 is more encompassing than its predecessor, you might notice higher charges for Cloud Composer environments while at the same time seeing lower costs for Compute Engine instances.

Autoscaling introduced in Cloud Composer 2 brings additional efficiency in resource utilization because environments no longer need to be continuously scaled for the peak. The extent of cost savings generated by autoscaling is highly dependent on the pattern of DAG runs and environment configuration.

In Cloud Composer 2, compared to Cloud Composer 1:

• Costs for your environments might vary because of environment autoscaling. You only pay for resources that are utilized by your environment.
• You do not pay additional costs for Compute Engine components of your environment. Cloud Composer 2 SKUs replace these costs.
• Even though Cloud Composer 2 environments rely on GKE Autopilot clusters, you are not charged a cluster management fee that is present in regular GKE Autopilot clusters.

Cloud Composer 1 pricing

What's next

• Get started with Cloud Composer.
• Read the Cloud Composer documentation.
• Try the Pricing calculator.
• Learn about Cloud Composer solutions and use cases.