“The future of serverless is about running your code without thinking about servers, and that future is already here.”
— Tim Wagner, creator of AWS Lambda
Summary
Cloud Run is the Terraform note for the serverless application layer: it defines the shared locals, Cloud Run service, Cloud Run jobs, VPC and secret wiring, and post-deployment verification steps that turn the Terraform-managed GCP foundation into runnable HTTP endpoints and batch workloads.
Shared runtime foundation
covers locals for registry paths and database connection data, usage-based billing context, required APIs, and the region and project assumptions that all Cloud Run resources depend on
Service configuration
covers google_cloud_run_v2_service, deletion protection, image selection, scaling, session affinity, public access boundaries, secret injection, and Direct VPC Egress for the dashboard service
Job configuration
covers google_cloud_run_v2_job, nested job templates, pipeline and setup jobs, timeouts, retries, lifecycle meta-arguments, import paths, and force-replacement boundaries
Operations and safety
Warnings: disabling deletion protection weakens production safety, mutable image tags make deployments non-deterministic, broad public access is risky, and several Cloud Run changes still force replacement or require careful region migration
Recommendations: size runtime limits for workload behavior rather than headline cost, deploy immutable image references, restrict ingress intentionally, use direct private connectivity for SQL access, and verify services, jobs, executions, and logs with gcloud after apply
Glossary
Cloud Run service
A long-running HTTP-serving Cloud Run resource that scales in response to incoming requests.
It matters because the dashboard workload in this note is modeled as a service rather than as a batch-oriented job.
Services are request-driven
A Cloud Run service is meant to stay available for inbound traffic, even if it scales to zero between bursts. That is a different execution model from a run-to-completion batch job.
Cloud Run job
A run-to-completion Cloud Run resource designed for batch or scheduled execution instead of for long-lived HTTP traffic.
It matters because the pipeline and setup workloads in this note are better expressed as jobs than as always-on services.
Jobs and services share platform pieces, not purpose
Cloud Run jobs and services use related infrastructure, but they differ in lifecycle, triggering model, and configuration shape. Picking the wrong one usually makes the workload harder to operate.
Direct VPC Egress
A Cloud Run networking mode that lets a service or job send traffic directly into a VPC without using a connector proxy layer.
It matters because the workloads in this note need private-path access to the SQL VM inside the Terraform-managed network.
Private connectivity still needs design
Direct egress solves network reachability, but firewall rules, subnet planning, and backend service exposure still determine whether the end-to-end connection is actually safe.
google_cloud_run_v2_service
The Terraform resource for managing a Cloud Run v2 service configuration.
It matters because the dashboard deployment is expressed through this resource, including its image, environment, scaling, ingress, and IAM-adjacent runtime configuration.
Some fields are operationally sensitive
A service resource looks compact in HCL, but changes to certain fields can cause new revisions, access changes, or replacement-like effects. The semantics of each field matter beyond basic syntax.
google_cloud_run_v2_job
The Terraform resource for defining a Cloud Run v2 job, including its nested execution template.
It matters because the job resource structure is deeper and less intuitive than a service resource, especially when you need retries, timeouts, or multiple containers.
Template nesting is easy to misread
Cloud Run jobs use a double-nested template structure in Terraform. If you lose track of which template block you are in, arguments end up on the wrong object surprisingly easily.
Deletion protection
A safeguard that prevents accidental deletion of a Cloud Run resource while enabled.
It matters because serverless resources can still be mission-critical, and a mistaken destroy should not be trivial in production.
Disabled protection lowers the floor
Setting deletion protection off may be fine in a lab, but it removes an important safety net for real workloads. Production defaults should usually bias toward protection.
Image digest / immutable image reference
A content-addressed image identifier that points to one exact container image build, unlike a mutable tag such as latest.
It matters because reproducible Cloud Run deployment depends on Terraform referencing a deterministic image artifact.
Mutable tags hide drift
If latest points to a different image tomorrow, Terraform may appear unchanged while the runtime behavior has drifted. Immutable references make deployments auditable.
Secret injection
The Cloud Run pattern of exposing Secret Manager values to containers through environment variables or mounted references instead of hardcoding secrets in the image.
It matters because the application layer in this note depends on runtime secrets without baking them into the container artifact.
Better than baking secrets into images
Keeping secrets external lets you rotate them without rebuilding application images and avoids leaving sensitive material in the registry layer history.
Session affinity
A service configuration option that biases subsequent requests from the same client toward the same serving instance.
It matters because some web workloads behave better when repeated requests stay near the same in-memory session state.
It is not a substitute for state design
Session affinity can reduce churn, but it does not turn Cloud Run into a stateful platform. Any truly durable session state still needs an external backing system.
Service account user / roles/iam.serviceAccountUser
The permission needed to attach a service account identity to a Cloud Run service or job.
It matters because Terraform deployments of serverless resources often fail unless the deployer can act as the runtime identity.
Deployment rights and runtime rights are separate
Being allowed to deploy a Cloud Run resource does not automatically mean you can attach any service account to it. That boundary is intentional and should stay explicit.
Revision
A versioned Cloud Run deployment snapshot created when certain service or job configuration changes are applied.
It matters because operational behavior, rollout history, and troubleshooting often depend on knowing which revision is actually serving traffic or executing jobs.
Many changes are revision-creating changes
Image updates, environment changes, and several runtime settings produce new revisions rather than mutating the old one in place. That makes Cloud Run feel more like deployment history than like direct resource mutation.
Import path
The fully qualified identifier Terraform needs to adopt an existing Cloud Run resource into state.
It matters because services and jobs are often created before Terraform takes ownership, and adoption needs the correct resource path shape.
Adoption is part of operations too
Terraform management does not always start on day one. Import paths let an already-running service or job become part of the reviewed infrastructure workflow later.
Billing is usage-based
Cloud Run bills actual CPU/memory usage, not the limits defined in the configuration. Setting cpu = "2" and memory = "2Gi" as limits does not mean you pay for 2 CPUs — you pay for what the container actually consumes during execution. Lowering limits does not save cost; it only risks OOM kills or CPU throttling if the workload exceeds them.
Assumed variables and prerequisites
All resource blocks in this file reference var.region and var.project_id, which must be defined in your variables file. The following GCP APIs must be enabled on the project:
run.googleapis.com — Cloud Run services and jobs
secretmanager.googleapis.com — secret injection into containers
compute.googleapis.com — VPC networking for Direct VPC Egress
The Terraform service account needs at minimum: roles/run.admin, roles/iam.serviceAccountUser (to attach service accounts to Cloud Run resources), and roles/secretmanager.secretAccessor (to read secrets at deploy time).
flowchart LR
A["Internet"] -->|HTTPS| B["Cloud Run Service<br/>dashboard"]
C["Airflow"] -->|execute| D["Cloud Run Job<br/>pipeline / setup"]
B -->|Direct VPC Egress| E["SQL VM<br/>10.0.0.x:1433"]
D -->|Direct VPC Egress| E
F["Secret Manager"] -.->|env injection| B
F -.->|env injection| D
G["Artifact Registry"] -.->|image pull| B
G -.->|image pull| D
Shared Configuration
Computed values reused across all Cloud Run resources in this file.
locals
Terraform locals are computed values evaluated once at plan time. They cannot be overridden from outside the module — use variable blocks for configurable inputs.
Compute the full registry path, SQL VM private IP, and SA username for reuse across Cloud Run resources.
Full registry path. Used as prefix for image references.
sql_ip
10.0.0.x (resolved at apply time)
The SQL VM’s private IP. Read from the VM’s first network interface. Used in connection strings.
sql_user
sa
SQL Server system administrator username.
google_cloud_run_v2_service
A Cloud Run service is a long-running HTTP endpoint that auto-scales based on incoming traffic. Unlike jobs, services stay alive to serve requests. This resource provisions the dashboard — a Blazor Server application serving the project’s web interface. For the architectural distinction between services and jobs, including when to choose each, see Cloud Run jobs vs services.
Declare the dashboard Cloud Run service with deletion protection disabled.
Service name. The public URL is derived from this: data-pipeline-dashboard-xxxxx-ew.a.run.app.
deletion_protection
false
When true, Terraform refuses to destroy this resource. Set to false here to allow teardown via terraform destroy. In production, consider true for databases.
deletion_protection = false
With deletion_protection = false, terraform destroy or removing the resource from config will immediately delete the Cloud Run service and all its revisions. Changing location also forces a destroy-and-recreate, which causes downtime and a new URL.
Production safeguard
Set deletion_protection = true for production services. To intentionally destroy, first set it to false, run terraform apply, then destroy. For stateful resources, also add lifecycle { prevent_destroy = true }.
Template Block
Configure sticky sessions, scaling limits, and a 1-hour timeout for Blazor WebSocket connections.
Sticky sessions — routes requests from the same client to the same container instance. Critical for Blazor Server, which maintains a persistent WebSocket (SignalR circuit) per user. Without this, WebSocket connections would break when routed to a different instance.
service_account
data-pipeline-dashboard@...
The identity the container runs as. Determines what GCP APIs it can call.
timeout
3600s
Maximum request duration (1 hour). Blazor’s WebSocket connections are long-lived — the default 300s would disconnect users after 5 minutes.
min_instance_count
1
Always-warm — at least one instance is always running. Eliminates cold start latency (which would break WebSocket connections). Costs ~$5-10/month for an idle instance.
max_instance_count
2
Limits scaling to 2 instances. This dashboard serves a small number of users — no need for aggressive autoscaling.
Container Block — Dashboard
Define the dashboard container with startup probe, connection string, secret injection, and resource limits.
Docker image to run. latest tag is updated by GitHub Actions on every push to main.
container_port
8080
Port the Blazor app listens on inside the container. Cloud Run routes external HTTPS traffic to this port.
startup_probe
HTTP GET /
Cloud Run checks if the container is ready by hitting / every 10 seconds, starting 3 seconds after launch. If it fails 3 times, the container is killed and restarted.
ConnectionStrings__project
ADO.NET connection string (without password)
.NET convention: double underscore __ maps to : in appsettings.json hierarchy. Equivalent to ConnectionStrings:data-pipeline. Contains the SQL VM’s private IP, database name, and user — but not the password. TrustServerCertificate=true skips SSL certificate validation (acceptable for internal VPC traffic).
DB_PASSWORD
Secret Manager ref
The database password is injected from Secret Manager at container startup. Program.cs reads this env var and merges it into the connection string via SqlConnectionStringBuilder. The password never appears in Terraform state or Cloud Run configuration.
cpu
1
1 vCPU allocated to the container.
memory
512Mi
512 megabytes of RAM. Sufficient for Blazor Server with a small number of concurrent circuits.
Non-deterministic image tags
Using :latest means terraform plan cannot detect image changes — the tag stays the same even when the underlying image is updated by CI/CD. Terraform will show “no changes” even after a new image is pushed.
Deterministic deployments
Use image digests (@sha256:...) or immutable version tags for full traceability. Alternatively, add lifecycle { ignore_changes = [template[0].containers[0].image] } if image updates are intentionally managed outside Terraform (e.g., by GitHub Actions).
VPC Access — Direct Egress
Route private-range traffic through the VPC via Direct VPC Egress.
Direct VPC egress — Cloud Run gets a network interface in the VPC, allowing it to reach private IPs (like the SQL VM at 10.0.0.x). This replaced the older VPC Connector approach.
egress
PRIVATE_RANGES_ONLY
Only traffic destined for private IP ranges (RFC 1918: 10.x, 172.16-31.x, 192.168.x) goes through the VPC. Public internet traffic (e.g., external API calls) uses Cloud Run’s default route. This prevents database traffic from ever touching the public internet.
Direct VPC Egress vs VPC Connector
Direct VPC Egress (used here) attaches a network interface directly to the VPC — no separate connector resource needed. This replaced the older google_vpc_access_connector approach, which required provisioning a dedicated /28 subnet and had throughput limits (up to 1 Gbps). Direct VPC Egress supports higher bandwidth, has no additional resource cost, and simplifies the Terraform configuration. If migrating from a VPC Connector, remove the connector resource and replace the vpc_access block with the network_interfaces syntax shown above.
google_cloud_run_v2_service_iam_member
Make the dashboard service publicly accessible without authentication.
resource "google_cloud_run_v2_service_iam_member" "dashboard_public" { name = google_cloud_run_v2_service.dashboard.name location = var.region role = "roles/run.invoker" member = "allUsers"}
Field
Value
Meaning
member
allUsers
A special IAM principal meaning “anyone on the internet.” This makes the dashboard publicly accessible without authentication. Without this binding, Cloud Run returns 403 to unauthenticated requests.
Public internet access
Binding allUsers with roles/run.invoker makes this service accessible to anyone on the internet without authentication. Any person or bot can send requests to the service URL. This is appropriate for a public dashboard but dangerous for internal tools or APIs that handle sensitive data.
Restrict access
For internal tools, use allAuthenticatedUsers (requires Google login) or specific service accounts and groups. For zero-trust access, use Identity-Aware Proxy (IAP) to enforce authentication at the load balancer level.
google_cloud_run_v2_job
A Cloud Run job runs a container to completion and exits. Unlike a service, it has no HTTP endpoint — it is triggered externally (by Airflow or the gcloud CLI). This section covers two job variants: the pipeline job (recurring data processing) and the setup job (one-time initialization).
Declare the pipeline Cloud Run job for batch data processing.
The outer template (execution template) controls how many parallel tasks to run. The inner template (task template) defines the container spec, service account, timeout, and retries. This structure exists because Cloud Run jobs support fan-out: if task_count = 10, it would spawn 10 identical containers in parallel. Each container receives a CLOUD_RUN_TASK_INDEX env var (0-9) to know which shard of work to handle.
Field
Value
Meaning
task_count
1
Number of parallel tasks per execution. Set to 1 because the pipeline handles all indices sequentially within a single process.
timeout
1800s
Maximum runtime (30 minutes). The full pipeline typically completes in 2-5 minutes. The generous timeout accommodates slow API responses or large backfills.
max_retries
1
If the container exits with a non-zero code, Cloud Run retries once. Handles transient failures (network blips, OOM). The pipeline is idempotent, so retrying is always safe.
Pipeline Job Environment Variables
Inject the database password from Secret Manager at container startup.
env { name = "SA_PASSWORD" value_source { secret_key_ref { secret = google_secret_manager_secret.db_password.secret_id version = "latest" } }}
Field
Value
Meaning
value_source.secret_key_ref
—
Secret injection — Cloud Run reads the secret from Secret Manager at container startup and injects it as an environment variable. The container never sees the secret in its configuration — only at runtime in memory.
version
latest
Always use the most recent version of the secret. Alternatively, you can pin to a specific version number for stability.
Other pipeline environment variables:
Variable
Value
Purpose
SQL_HOST
local.sql_ip
SQL VM’s private IP
SQL_PORT
1433
SQL Server port
SQL_DATABASE
data-pipeline
Database name
SQL_USER
sa
SQL Server admin
DD_SERVICE
data-pipeline-pipeline
Datadog service name for APM traces
DD_ENV
prod
Datadog environment tag
DD_TRACE_AGENT_URL
http://<airflow-ip>:8126
APM trace endpoint on Airflow VM
DD_API_KEY
(from Secret Manager)
Datadog API key for direct log shipping
LOG_FORMAT
json
Structured JSON logs for Datadog parsing
google_cloud_run_v2_job | Setup
The setup job runs one-time initialization tasks — creating database schemas, seeding reference data, and downloading historical data. It uses the same container image as the pipeline job but overrides the entrypoint with a custom command.
Declare the setup job with a custom entrypoint for one-time database initialization.
Entrypoint override — runs DDL scripts (create schemas/tables) then sets up all indices (fetch dimensions, seed history). && ensures the second command only runs if the first succeeds.
max_retries
0
No automatic retries. Setup is a one-time operation — if it fails, investigate the logs rather than blindly retrying.
timeout
3600s
1 hour. Initial setup includes downloading historical data for all configured instruments — this can take 10-15 minutes.
Lifecycle meta-arguments for Cloud Run
ignore_changes: Add lifecycle { ignore_changes = [template[0].containers[0].image] } to services and jobs whose image tag is updated by CI/CD outside of Terraform. This prevents Terraform from reporting drift on every plan.
prevent_destroy: Set lifecycle { prevent_destroy = true } on production services to block accidental deletion via terraform destroy.
create_before_destroy: Cloud Run creates a new revision before routing traffic away from the old one by default, so this meta-argument is rarely needed at the Terraform level.
Force-replacement triggers
Changing location on a google_cloud_run_v2_service or google_cloud_run_v2_job forces Terraform to destroy and recreate the resource (# forces replacement in plan output). For services, this means the public URL changes and all traffic is interrupted. For jobs, in-progress executions are terminated.
Safe region migration
To migrate a Cloud Run resource to a new region: deploy the new resource alongside the old one (with a different Terraform resource name), migrate traffic or triggers, verify the new resource works, then remove the old resource from config.
Import existing Cloud Run resources
To bring an existing Cloud Run service or job under Terraform management:
Add the resource block to your .tf file matching the current configuration