Google Cloud Workflows: What It Is, Why to Use It, and 5 Real Code Examples

Most real applications do not do one job. A single customer order can touch a stock service, a payment service, a billing service and a notification service. Someone has to decide which service runs first, what happens when one fails, and how to retry it.

Google Cloud Workflows does this job. You write the steps in a simple YAML file, and Google Cloud runs them in order, keeps track of progress, and handles retries for you.

In this guide, you will learn what Workflows is, what problem it solves, when to use it, and how to use it through 5 detailed code examples.

What Is Google Cloud Workflows?

Google Cloud Workflows is a fully managed, serverless orchestration service. Orchestration means controlling many services so they work together in the correct order.

Think of it like a project manager. The manager does not write the code or build the product. The manager tells each team when to start, checks the result, and decides what happens next. Your Cloud Run services, BigQuery jobs and APIs do the real work, and Workflows manages the order.

Workflows can call:

• Cloud Run services and Cloud Run functions
• Google Cloud services such as BigQuery, Cloud Storage and Firestore
• Any HTTP-based API, including APIs outside Google Cloud

What Problem Does Workflows Solve?

Without an orchestrator, developers usually write the control logic inside the application code. One service calls the next, which calls the next. This creates several problems:

• Tight coupling: Changing the order of steps means changing and redeploying several services.
• Repeated retry code: Every service needs its own logic for handling failures.
• Lost progress: If a process stops halfway, it is hard to know which steps finished.
• Difficult waiting: A normal function cannot sit idle for hours waiting for a manager’s approval.
• Poor visibility: When something fails, you have to search through the logs of many services.

Workflows moves this control logic into one file. The services stay simple, and the full process is easy to read in one place.

What Can Workflows Do?

• Run steps one after another
• Make decisions with conditions
• Store data in variables and pass it between steps
• Retry failed calls with exponential backoff
• Catch errors and run backup steps
• Run steps in parallel to save time
• Pause and wait for an outside event or a human decision
• Use built-in connectors for Google Cloud services
• Hold state, retry, poll or wait for up to one year in a single run

What Is the Use of Workflows?

Google describes four main uses:

• Service orchestration: Combine services into one solution. For example, send an uploaded image to Vision AI and save its tags in Firestore.
• Batch jobs: Process many items on a schedule, such as daily reports or customer emails.
• Business processes: Automate approval flows that need a human decision.
• IT automation: Script Google Cloud tasks, such as creating projects or shutting down resources on a schedule.

When to Use Workflows

Use Workflows when:

• You need to connect several APIs or microservices in a fixed order.
• You need retries, error handling or long waits without writing that code yourself.
• You run simple BigQuery or Cloud Storage tasks in sequence.
• You want a serverless tool that costs nothing while it sits idle.

Do not use it for heavy computing or complex business logic. Workflows should call services and handle their responses. Keep large calculations and data processing inside Cloud Run, BigQuery or another service.

Why Use Workflows?

• No servers to manage: Google runs and scales everything.
• No cost when idle: You pay for the steps that run.
• Easy to change: Update one YAML file instead of several services.
• Clear history: You can review each execution step by step.
• Nothing to patch: A workflow has no code libraries to keep updated.

Workflows vs Cloud Composer

Beginners often confuse these two services. Both are orchestrators, but they are built for different work.

Point	Workflows	Cloud Composer
Best for	Connecting services and APIs	Data-driven batch pipelines
Written in	YAML or JSON	Python DAGs
Based on	Google managed service	Apache Airflow
Speed between steps	Milliseconds	A few seconds is acceptable
Special strength	Scales down to zero	Backfills and DAG re-runs

Simple rule: Choose Workflows to connect services. Choose Cloud Composer for large data pipelines that need Airflow features.

How Workflows Works in Real Time

Imagine an online shop. A customer places an order, and the shop must complete these tasks:

1. Check the stock.
2. Take the payment. If the payment API is busy, try again.
3. Create the invoice.
4. If the refund amount is high, wait for a manager’s approval.
5. At night, load the sales file into BigQuery for reports.

The examples below build these exact tasks. They use a sample shop, so replace the URLs, bucket, dataset and table names with your own.

Setup Before You Start

First, turn on the APIs and create a service account for your workflows. Replace PROJECT_ID with your project ID.

gcloud services enable workflows.googleapis.com workflowexecutions.googleapis.com

gcloud iam service-accounts create wf-runner --display-name="Workflows runner"

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:wf-runner@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/logging.logWriter"

Give this service account only the roles it needs. To call private Cloud Run services, add roles/run.invoker. For the BigQuery examples, add roles/bigquery.jobUser and roles/bigquery.dataEditor. Example 3 also needs roles/storage.objectViewer.

Use these commands to deploy and run any example:

gcloud workflows deploy order-flow \
  --source=order-flow.yaml \
  --location=asia-south1 \
  --service-account=wf-runner@PROJECT_ID.iam.gserviceaccount.com

gcloud workflows run order-flow \
  --location=asia-south1 \
  --data='{"orderId":"A1001"}'

Example 1: Order Processing with a Condition

Goal: Read an order and check the stock. If stock is available, create an invoice. If not, put the order on hold.

main:
  params: [input]
  steps:
    - get_order:
        call: http.get
        args:
          url: ${"https://orders-api-xxxxx-el.a.run.app/orders/" + input.orderId}
          auth:
            type: OIDC
        result: order
    - check_stock:
        switch:
          - condition: ${order.body.quantity <= order.body.stock}
            next: create_invoice
        next: put_on_hold
    - create_invoice:
        call: http.post
        args:
          url: https://billing-api-xxxxx-el.a.run.app/invoices
          auth:
            type: OIDC
          body:
            orderId: ${input.orderId}
            amount: ${order.body.amount}
        result: invoice
    - done:
        return:
          status: "CONFIRMED"
          invoiceId: ${invoice.body.invoiceId}
    - put_on_hold:
        return:
          status: "ON_HOLD"
          reason: "Not enough stock"

How it works:

• params receives the input you send when the workflow starts.
• auth with OIDC lets the workflow call a private Cloud Run service securely.
• result saves the API response in a variable. The JSON response is available under .body.
• switch checks the condition and next jumps to the correct step.

Example 2: Payment with Retry and Error Handling

Goal: Call a payment API that sometimes fails for a short time. Retry safely, and log the error if every attempt fails.

main:
  params: [input]
  steps:
    - take_payment:
        try:
          call: http.post
          args:
            url: https://payments-api-xxxxx-el.a.run.app/charge
            auth:
              type: OIDC
            headers:
              Idempotency-Key: ${input.orderId}
            body:
              orderId: ${input.orderId}
              amount: ${input.amount}
            timeout: 30
          result: payment
        retry:
          predicate: ${http.default_retry_predicate}
          max_retries: 5
          backoff:
            initial_delay: 2
            max_delay: 60
            multiplier: 2
        except:
          as: e
          steps:
            - log_failure:
                call: sys.log
                args:
                  severity: ERROR
                  text: ${"Payment failed for " + input.orderId + " - " + e.message}
            - stop_with_error:
                raise: ${e}
    - finish:
        return:
          status: "PAID"
          paymentId: ${payment.body.paymentId}

How it works:

• The workflow waits 2 seconds before the first retry, then doubles the wait each time, up to 60 seconds.
• http.default_retry_predicate decides which errors are temporary and worth retrying.
• except runs after all retries fail, or when the error should not be retried.
• raise marks the execution as failed, so the failure is visible in the execution history.

Important: A retry can send the same payment twice. Your payment service must read the Idempotency-Key header and ignore a repeated request. Without this check, a customer could be charged twice.

Example 3: Load a CSV File into BigQuery

Goal: Every night, load the sales file from Cloud Storage into a staging table, check that it has data, and merge it into the final table.

main:
  steps:
    - init:
        assign:
          - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
          - today: ${text.substring(time.format(sys.now(), "Asia/Kolkata"), 0, 10)}
          - file_uri: ${"gs://my-sales-bucket/incoming/sales_" + today + ".csv"}
    - load_to_staging:
        call: googleapis.bigquery.v2.jobs.insert
        args:
          projectId: ${project_id}
          body:
            configuration:
              load:
                sourceUris:
                  - ${file_uri}
                destinationTable:
                  projectId: ${project_id}
                  datasetId: shop
                  tableId: sales_stg
                sourceFormat: CSV
                skipLeadingRows: 1
                autodetect: true
                writeDisposition: WRITE_TRUNCATE
    - count_rows:
        call: googleapis.bigquery.v2.jobs.query
        args:
          projectId: ${project_id}
          body:
            useLegacySql: false
            query: SELECT COUNT(*) FROM shop.sales_stg
        result: count_result
    - check_rows:
        switch:
          - condition: ${int(count_result.rows[0].f[0].v) == 0}
            steps:
              - stop_load:
                  raise: "Staging table is empty, stopping the load"
    - merge_to_final:
        call: googleapis.bigquery.v2.jobs.query
        args:
          projectId: ${project_id}
          body:
            useLegacySql: false
            query: |
              MERGE shop.sales AS target
              USING shop.sales_stg AS source
              ON target.sale_id = source.sale_id
              WHEN MATCHED THEN
                UPDATE SET amount = source.amount, status = source.status
              WHEN NOT MATCHED THEN
                INSERT (sale_id, amount, status)
                VALUES (source.sale_id, source.amount, source.status)
    - finish:
        return: ${"Loaded " + file_uri}

How it works:

• The workflow builds the file name from today’s date, for example sales_2026-11-04.csv.
• googleapis.bigquery.v2.jobs.insert is a connector. It handles authentication for you.
• WRITE_TRUNCATE replaces the staging table on every run.
• The row check stops the workflow early if the file is empty.
• MERGE updates existing sales and inserts new ones, so a repeat run does not create duplicates in the final table.

To run this workflow every night at 2:00 AM India time, create a Cloud Scheduler job. The service account below needs the roles/workflows.invoker role.

gcloud scheduler jobs create http daily-sales-load \
  --location=asia-south1 \
  --schedule="0 2 * * *" \
  --time-zone="Asia/Kolkata" \
  --uri="https://workflowexecutions.googleapis.com/v1/projects/PROJECT_ID/locations/asia-south1/workflows/sales-elt/executions" \
  --http-method=POST \
  --oauth-service-account-email=wf-runner@PROJECT_ID.iam.gserviceaccount.com \
  --message-body='{}'

Example 4: Check Five Tables in Parallel

Goal: Count the rows in five BigQuery tables at the same time. If each query takes 10 seconds, running them one by one takes about 50 seconds. Running them in parallel takes about 10 seconds.

main:
  steps:
    - init:
        assign:
          - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
          - tables: ["orders", "customers", "payments", "refunds", "shipments"]
          - results: {}
    - count_all_tables:
        parallel:
          shared: [results]
          for:
            value: table
            in: ${tables}
            steps:
              - count_table:
                  try:
                    steps:
                      - run_query:
                          call: googleapis.bigquery.v2.jobs.query
                          args:
                            projectId: ${project_id}
                            body:
                              useLegacySql: false
                              query: ${"SELECT COUNT(*) FROM shop." + table}
                          result: query_result
                      - save_count:
                          assign:
                            - results[table]: ${query_result.rows[0].f[0].v}
                  except:
                    as: e
                    steps:
                      - save_error:
                          assign:
                            - results[table]: ${"FAILED - " + e.message}
    - finish:
        return: ${results}

How it works:

• parallel with for runs each loop iteration at the same time.
• shared lets every iteration write to the results variable.
• try and except save an error against one table, so the other tables can still report their counts.

One execution can run 20 branches and iterations at once. Any extra ones wait in a queue.

Example 5: Manager Approval with a Callback

Goal: Pause a refund until a manager approves it. If nobody replies within 24 hours, stop the process.

main:
  params: [input]
  steps:
    - create_callback:
        call: events.create_callback_endpoint
        args:
          http_callback_method: "POST"
        result: callback_details
    - ask_manager:
        call: http.post
        args:
          url: https://notify-api-xxxxx-el.a.run.app/approval-request
          auth:
            type: OIDC
          body:
            refundId: ${input.refundId}
            amount: ${input.amount}
            approvalUrl: ${callback_details.url}
    - wait_for_manager:
        try:
          call: events.await_callback
          args:
            callback: ${callback_details}
            timeout: 86400
          result: callback_request
        except:
          as: e
          steps:
            - no_reply:
                return:
                  status: "EXPIRED"
                  reason: ${e.message}
    - check_decision:
        switch:
          - condition: ${callback_request.http_request.body.decision == "APPROVED"}
            next: send_refund
        next: reject_refund
    - send_refund:
        call: http.post
        args:
          url: https://payments-api-xxxxx-el.a.run.app/refund
          auth:
            type: OIDC
          body:
            refundId: ${input.refundId}
            amount: ${input.amount}
    - refund_done:
        return:
          status: "REFUNDED"
    - reject_refund:
        return:
          status: "REJECTED"

Your approval app resumes the workflow by calling the callback URL with an access token:

curl -X POST "CALLBACK_URL" \
  -H "Authorization: Bearer ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"decision":"APPROVED"}'

How it works:

• events.create_callback_endpoint creates a unique URL for this execution.
• events.await_callback pauses the workflow without polling. The timeout of 86400 seconds equals 24 hours.
• The caller needs the workflows.callbacks.send permission, which is part of the Workflows Invoker role.
• The manager should click a button in your app. Your backend then calls the URL with its own token, so no token reaches the browser.

Workflows Pricing

Workflows charges by the number of steps. Successful steps, failed steps and each retry all count. Check the official pricing page for the latest prices.

Step type	Free each month	After the free tier
Internal steps	5,000	$0.01 per 1,000 steps
External steps	2,000	$0.025 per 1,000 steps

Internal steps include calls to Google Cloud APIs, Cloud Run calls, variable assignments and condition checks. External steps include calls to outside APIs and steps that wait for a callback.

Important Limits

Limit	Value
Maximum run time for one execution	1 year
Workflow source file size	128 KB
Memory for variables in one execution	512 KB
Size of one HTTP response	2 MB
Branches in one parallel step	10
Branches and iterations running at once	20

The 512 KB memory limit is the one most people hit first. Do not load large query results into variables. Keep large data in BigQuery or Cloud Storage, and pass only IDs or file paths between steps.

Best Practices

• Use a separate service account with only the roles the workflow needs.
• Make retried steps idempotent, so running one twice causes no harm.
• Use connectors for Google Cloud services instead of raw HTTP calls.
• Use callbacks instead of long polling loops.
• Log important values with sys.log, but never log passwords, tokens or card details.
• Keep business logic in your services and control logic in your workflow.

Conclusion

Google Cloud Workflows gives you one simple place to control your services. It runs your steps in order, retries temporary failures, runs tasks in parallel, and waits for events or human decisions without any servers to manage.

Start with Example 1, deploy it in a test project, and open the execution history to see each step. Then add retries, BigQuery steps and callbacks as your project needs them.

FAQs

Q1: Do I need to write Python or Java to use Workflows?
No. You write a workflow in YAML or JSON. Your services can use any language.

Q2: Can Workflows start automatically?
Yes. Cloud Scheduler can start a workflow on a schedule, and Eventarc can start one when an event happens, such as a file upload.

Q3: Is Workflows the same as Cloud Scheduler?
No. Cloud Scheduler decides when a job starts. Workflows controls the steps after it starts.

Q4: Can Workflows call an API outside Google Cloud?
Yes. It can call any HTTP-based API. These calls count as external steps.

Q5: Is Workflows good for processing big data?
No. It should control the job, not process the data itself. Let BigQuery or another service do the heavy work.