03 - GCP Total Cost of Ownership

22 min read

GCP Total Cost of Ownership

Why This Matters

Static architecture estimates age badly because they are built from imaginary volumes and stale rate cards. A project-grounded TCO note should start from what actually exists, then separate what is measured, what is priced, and what is still blocked by missing billing export. In bq-wh-nb, that distinction changes the answer completely: the steady monthly floor is driven by VMs and disks, not by BigQuery queries or Cloud Run jobs.

Conceptual Model

TCO is easiest to reason about when you split the platform into fixed, variable, and blocked or unverified components.


flowchart TD
    F["Fixed drivers<br>VM runtime · disks · base networking"] --> T["Project TCO"]
    V["Variable drivers<br>BigQuery scans · Cloud Run runtime · logs · storage growth"] --> T
    B["Blocked or unverified here<br>billing export absent · taxes · credits · negotiated pricing"] --> T

Archived Inventory Of Cost Drivers

The archived project was small enough that the resource inventory was still the most honest starting point for TCO.

PowerShell / Linux | gcloud | inventory always-on infrastructure

These commands identify the recurring drivers that continue to exist even if no new analytical work is executed today.

List running Compute Engine instances

Run this at the start of any TCO or rightsizing review. It is typically triggered by use it when you need to know which workloads create the platform’s fixed compute floor. This is a read-only Compute Engine inventory query. Identify always-on machine types, regions, and labels.

FieldSource columnUnit / typeMeaning
nameInstance metadataSTRINGVM name.
zoneInstance metadataSTRINGZone hosting the VM.
machineType.basename()Instance metadataSTRINGMachine type that drives compute pricing.
statusInstance metadataSTRINGWhether the instance is running and therefore accruing compute charges.
labelsInstance metadataMAPAttribution metadata currently applied to the instance.

This command lists the current Compute Engine instances in bq-wh-nb.

gcloud compute instances list --format="table(name,zone,machineType.basename(),status,labels)"
namezonemachineType.basename()statuslabels
stoxx-airfloweurope-west1-be2-standard-2RUNNING{'app': 'stoxx-airflow', 'env': 'dev'}
stoxx-vmeurope-west1-be2-mediumRUNNING{'app': 'stoxx-db', 'env': 'dev'}

Both VMs are currently running, so both contribute to the project’s steady compute floor.

List attached persistent disks

Run this whenever you compare stop-vs-delete savings or model storage-heavy VM architectures. It is typically triggered by use it when VM costs look modest but the monthly bill still does not drop after stoppage. This is a read-only persistent-disk inventory query. Quantify attached disk sizes and types, which continue to bill independently of instance runtime.

FieldSource columnUnit / typeMeaning
nameDisk metadataSTRINGPersistent disk name.
sizeGbDisk metadataINTEGER GiBProvisioned disk size.
type.basename()Disk metadataSTRINGDisk class used for pricing.
users.len()Computed inventory fieldINTEGERNumber of attached users; 1 means the disk is attached.

This command lists current persistent disks in the project.

gcloud compute disks list --format="table(name,zone,sizeGb,type.basename(),status,users.len())"
namezonesizeGbtype.basename()statususers.len()
stoxx-airfloweurope-west1-b30pd-balancedREADY1
stoxx-dataeurope-west1-b100pd-ssdREADY1
stoxx-logeurope-west1-b20pd-ssdREADY1
stoxx-tempdbeurope-west1-b20pd-ssdREADY1
stoxx-vmeurope-west1-b50pd-balancedREADY1

The disk inventory is important because attached disks continue to bill even if the VMs are stopped.

Measure bucket footprint

Run this before treating object storage as a meaningful TCO driver. It is typically triggered by use it when a design review assumes GCS is a large contributor to monthly cost. This is a read-only aggregate size query across the main workload buckets. Separate real storage cost drivers from buckets that are operationally present but financially negligible.

FieldSource columnUnit / typeMeaning
First columngcloud storage du -s resultINTEGER bytesTotal stored bytes in the bucket.
Second columngcloud storage du -s resultSTRINGBucket URI.

This command measures the current footprint of the main workload buckets.

gcloud storage du -s gs://stoxx-bq-bucket gs://stoxx-sql-bucket gs://stoxx-stage-bucket
20540 gs://stoxx-bq-bucket
100597760 gs://stoxx-sql-bucket
1193638 gs://stoxx-stage-bucket

Current GCS footprint is tiny. Separate gcloud storage buckets describe calls also show these buckets are STANDARD, regional EUROPE-WEST1, and currently protected by a 7-day soft-delete policy (retentionDurationSeconds = 604800).

FlagSyntaxDescription
--format--format="table(...)"Projects only the fields needed for TCO inspection.
-sgcloud storage du -sReturns per-bucket totals instead of per-object output.

PowerShell / Linux | BigQuery / Cloud Run | measure variable workload

Variable drivers are the easiest place to make wrong assumptions. The archived query and execution history shows that workload volume was still very small.

Summarize BigQuery query activity by principal

Run this when you need to know whether analytical workloads are large enough to matter in the current TCO. It is typically triggered by use it during warehouse design reviews and before considering BigQuery editions. This is a read-only SQL query against JOBS_BY_PROJECT. Measure current query activity without pretending it is the same as billed cost.

FieldSource columnUnit / typeMeaning
user_emailJOBS_BY_PROJECT.user_emailSTRINGPrincipal that submitted the queries.
query_countCOUNT(*)INTEGERNumber of query jobs in the time window.
total_bytes_processedSUM(total_bytes_processed)INTEGER bytesLogical bytes processed by those queries.
total_slot_msSUM(total_slot_ms)INTEGER msAggregate slot time used by those queries.

This query summarizes current BigQuery analytical activity for the last 30 days.

SELECT
  user_email,
  COUNT(*) AS query_count,
  SUM(total_bytes_processed) AS total_bytes_processed,
  SUM(total_slot_ms) AS total_slot_ms
FROM `region-europe-west1`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
  AND job_type = 'QUERY'
  AND state = 'DONE'
GROUP BY user_email
ORDER BY total_bytes_processed DESC
LIMIT 20
user_emailquery_counttotal_bytes_processedtotal_slot_ms
alexper.recovery@gmail.com572013468312531
bq-wh-sa@bq-wh-nb.iam.gserviceaccount.com8583859334117943
github-actions-sa@bq-wh-nb.iam.gserviceaccount.com12616435296

The current analytical volume is far below any level that would justify BigQuery slots or capacity commitments.

List Cloud Run jobs

Run this before assuming serverless batch work is a major cost driver. It is typically triggered by use it during TCO reviews or when batch jobs are blamed for unexpected spend. This is a read-only Cloud Run inventory call scoped to europe-west1. Identify job-level CPU, memory, and execution history.

FieldSource columnUnit / typeMeaning
nameJob metadataSTRINGCloud Run job name.
template.template.containers[0].resources.limits.cpuJob metadataSTRINGCPU limit per task.
template.template.containers[0].resources.limits.memoryJob metadataSTRINGMemory limit per task.
executionCountJob metadataINTEGERNumber of recorded executions for the job.

This command lists current Cloud Run jobs in europe-west1.

gcloud run jobs list --region=europe-west1 --format=json
[
  {
    "name": "stoxx-bronze-load",
    "cpu": "2",
    "memory": "2Gi",
    "executionCount": 1
  },
  {
    "name": "stoxx-stage-fetch",
    "cpu": "2",
    "memory": "2Gi",
    "executionCount": 4
  }
]

The current job estate is tiny: two batch jobs, both with modest resource limits.

Inspect recent Cloud Run execution durations

Run this when you need a real runtime input for Cloud Run TCO. It is typically triggered by use it after job design changes or when a supposedly cheap batch pattern starts to look expensive. This is a read-only execution-history query for a single Cloud Run job. Measure actual runtime rather than assuming duration.

FieldSource columnUnit / typeMeaning
nameExecution metadataSTRINGIndividual execution name.
completionTimeExecution metadataTIMESTAMPWhen the execution finished.
completionStatusExecution metadataSTRINGWhether the execution finished successfully.
durationExecution metadataSTRINGWall-clock runtime of the execution.

This command returns recent executions for stoxx-stage-fetch.

gcloud run jobs executions list --job=stoxx-stage-fetch --region=europe-west1 --format=json
[
  { "name": "stoxx-stage-fetch-s2b48", "duration": "57.01s", "completionStatus": "completed successfully" },
  { "name": "stoxx-stage-fetch-xtrwd", "duration": "1m10.96s", "completionStatus": "completed successfully" },
  { "name": "stoxx-stage-fetch-rb465", "duration": "48.77s", "completionStatus": "completed successfully" },
  { "name": "stoxx-stage-fetch-tw826", "duration": "58.75s", "completionStatus": "completed successfully" }
]

The execution history confirms that current serverless batch work is measured in seconds, not hours.

FlagSyntaxDescription
--region--region=europe-west1Scopes Cloud Run inventory and execution history to the active region.
--job--job=stoxx-stage-fetchSelects the execution history for one Cloud Run job.
--format--format=jsonPreserves fields needed for later cost calculations.

PowerShell | Cloud Billing Catalog API | translate inventory into current list prices

The goal is not to recreate the invoice. The goal is to map archived inventory to current public prices in the same currency as the billing account.

Query current Compute Engine and disk prices

Run this when a TCO model needs current list prices rather than copied documentation values. It is typically triggered by use it during baseline refreshes or before any price-sensitive design decision. This is a read-only Pricing API call using the active gcloud token. Retrieve the rates that match the actual VM and disk types present in the project region.

FieldSource columnUnit / typeMeaning
descriptionSKU metadataSTRINGPriced behavior exposed by the SKU.
tieredRates.unitPricePricing expressionMONEYPublic list price for the SKU.
effectiveTimePricing metadataTIMESTAMPWhen that price became effective.

This PowerShell pipeline reads current list prices for E2 compute and the persistent-disk classes used by bq-wh-nb.

$token = gcloud auth print-access-token
$headers = @{ Authorization = "Bearer $token" }
$service = "6F81-5844-456A"
$pageToken = $null
$skus = @()
 
do {
  $url = "https://cloudbilling.googleapis.com/v1/services/$service/skus?pageSize=5000&currencyCode=CZK"
  if ($pageToken) { $url += "&pageToken=$pageToken" }
  $response = Invoke-RestMethod -Headers $headers -Uri $url
  $skus += $response.skus
  $pageToken = $response.nextPageToken
} while ($pageToken)
 
$skus |
  Where-Object {
    ($_.serviceRegions -contains "europe-west1") -and (
      $_.description -eq "E2 Instance Core running in EMEA" -or
      $_.description -eq "E2 Instance Ram running in EMEA" -or
      $_.description -eq "Balanced PD Capacity" -or
      $_.description -eq "SSD backed PD Capacity"
    )
  } |
  Select-Object description,
    @{n="tieredRates";e={$_.pricingInfo[0].pricingExpression.tieredRates}},
    @{n="effectiveTime";e={$_.pricingInfo[0].effectiveTime}} |
  ConvertTo-Json -Depth 8
[
  {
    "description": "E2 Instance Core running in EMEA",
    "tieredRates": { "startUsageAmount": 0, "unitPrice": { "currencyCode": "CZK", "units": "0", "nanos": 509871109 } },
    "effectiveTime": "2026-04-13T07:00:00Z"
  },
  {
    "description": "E2 Instance Ram running in EMEA",
    "tieredRates": { "startUsageAmount": 0, "unitPrice": { "currencyCode": "CZK", "units": "0", "nanos": 68343520 } },
    "effectiveTime": "2026-04-13T07:00:00Z"
  },
  {
    "description": "Balanced PD Capacity",
    "tieredRates": { "startUsageAmount": 0, "unitPrice": { "currencyCode": "CZK", "units": "2", "nanos": 125050000 } },
    "effectiveTime": "2026-04-13T07:00:00Z"
  },
  {
    "description": "SSD backed PD Capacity",
    "tieredRates": { "startUsageAmount": 0, "unitPrice": { "currencyCode": "CZK", "units": "3", "nanos": 612585000 } },
    "effectiveTime": "2026-04-13T07:00:00Z"
  }
]

These prices are enough to build a current fixed-cost baseline for the existing VMs and disks.

Calculate the recurring monthly baseline from archived inventory

Run this when you need a quick recurring monthly floor from the archived project state. It is typically triggered by use it after any machine-type or disk-size change. This is a local PowerShell calculation that uses current rates and archived inventory values already verified in this note. Convert the project’s fixed infrastructure into a monthly public list-price baseline in CZK.

FieldSource columnUnit / typeMeaning
componentLocal calculation outputSTRINGResource family being priced.
monthlyCzkLocal calculation outputDECIMALMonthly list-price estimate for that component.

This PowerShell snippet calculates the recurring monthly floor from the archived VM and disk inventory.

$vmCoreRate = 0.509871109
$vmRamRate = 0.06834352
$pdBalanced = 2.12505
$pdSsd = 3.612585
$hours = 730
 
$items = @()
$items += [pscustomobject]@{
  component = "stoxx-airflow vm"
  monthlyCzk = [math]::Round($hours * ((2 * $vmCoreRate) + (8 * $vmRamRate)), 4)
}
$items += [pscustomobject]@{
  component = "stoxx-vm vm"
  monthlyCzk = [math]::Round($hours * ((1 * $vmCoreRate) + (4 * $vmRamRate)), 4)
}
$items += [pscustomobject]@{
  component = "pd-balanced disks (80 GiB)"
  monthlyCzk = [math]::Round(80 * $pdBalanced, 4)
}
$items += [pscustomobject]@{
  component = "pd-ssd disks (140 GiB)"
  monthlyCzk = [math]::Round(140 * $pdSsd, 4)
}
 
$items | ConvertTo-Json -Depth 4
[
  { "component": "stoxx-airflow vm", "monthlyCzk": 1143.538 },
  { "component": "stoxx-vm vm", "monthlyCzk": 571.769 },
  { "component": "pd-balanced disks (80 GiB)", "monthlyCzk": 170.004 },
  { "component": "pd-ssd disks (140 GiB)", "monthlyCzk": 505.7619 }
]

This is the most important result in the note. It shows that the steady-state floor is driven by VMs and disks, not by serverless or analytical activity. The stoxx-vm calculation uses the shared-core e2-medium shape as the half-sized counterpart to e2-standard-2, which matches the captured machine-type metadata and keeps the estimate aligned with the actual VM family.

Calculate the current observed variable footprint

Run this when you want to check whether variable drivers are still negligible or starting to compete with the fixed floor. It is typically triggered by use it during monthly reviews or after workload growth. This is a local PowerShell calculation using archived execution durations, archived query bytes, and current list prices already verified in this note. Estimate the archived variable footprint for Cloud Run jobs and BigQuery analysis.

FieldSource columnUnit / typeMeaning
totalSecondsLocal calculation outputDECIMAL secondsSummed Cloud Run execution duration.
totalCzkLocal calculation outputDECIMALCurrent list-price estimate for the measured variable workload.
totalTiBLocal calculation outputDECIMAL TiBBigQuery bytes processed converted to tebibytes.
listPriceCzkIfAboveFreeTierLocal calculation outputDECIMALWhat the measured BigQuery workload would cost above the free tier.

This PowerShell snippet turns the archived Cloud Run and BigQuery activity into a variable-cost estimate.

$jobCpuRate = 0.000382509
$jobMemRate = 0.000042501
$totalSeconds = 57.01 + 70.96 + 48.77 + 58.75 + 31.19
$bytes = 201346831 + 83859334 + 616435
$analysisRate = 159.37875
 
[pscustomobject]@{
  cloudRun = [pscustomobject]@{
    totalSeconds = $totalSeconds
    totalCzk = [math]::Round(($totalSeconds * 2 * $jobCpuRate) + ($totalSeconds * 2 * $jobMemRate), 6)
  }
  bigQuery = [pscustomobject]@{
    totalBytes = $bytes
    totalTiB = [math]::Round($bytes / 1099511627776, 9)
    listPriceCzkIfAboveFreeTier = [math]::Round(($bytes / 1099511627776) * $analysisRate, 6)
  }
} | ConvertTo-Json -Depth 6
{
  "cloudRun": {
    "totalSeconds": 266.68,
    "totalCzk": 0.226683
  },
  "bigQuery": {
    "totalBytes": 285822600,
    "totalTiB": 0.000259954,
    "listPriceCzkIfAboveFreeTier": 0.041431
  }
}

Current variable workload is effectively negligible compared to the steady VM and disk floor. BigQuery usage is far below the first 1 TiB analysis tier, and the measured Cloud Run executions amount to only a fraction of one CZK.

FlagSyntaxDescription
currencyCode?currencyCode=CZKKeeps pricing aligned with the billing-account currency.
pageSize?pageSize=5000Retrieves a large enough SKU slice to filter locally by description and region.
ConvertTo-Json -Depth 8ConvertTo-Json -Depth 8Preserves nested tiered-rate structures in the output.

Archived bq-wh-nb TCO Baseline

The archived evidence points to a very clear hierarchy of cost drivers.

DriverArchived basisApprox monthly list price (CZK)Interpretation
stoxx-airflow VMe2-standard-2, running1143.538Largest fixed compute driver in the project.
stoxx-vm VMe2-medium, running571.769Second fixed compute driver.
pd-balanced disks80 GiB total170.004Persists whether or not the VMs are busy.
pd-ssd disks140 GiB total505.7619Meaningful fixed storage floor.
Cloud Run jobs266.68 seconds observed0.226683Operationally real but financially tiny right now.
BigQuery analysis285822600 bytes over 30 days0.041431 above free tierCurrent usage is still tiny and effectively free at public-tier thresholds.
Main GCS bucketsAbout 0.095 GiB totalabout 0.0403Not a material cost driver today.

The current recurring list-price floor for just the two VMs and five attached disks is about 2391.0729 CZK per month before free-tier effects, taxes, credits, or negotiated pricing. That is the number that matters most for this project’s current architecture.

Current product note: TCO still needs effective-cost evidence

Current Cloud Billing documentation is clearer about the gap between inventory-times-list-price math and what the account actually pays. Billing export, price table reporting, invoice rows, and commitment metadata are still the required sources when TCO needs to move from engineering estimate to finance-grade reconciliation.

There are also important costs that remain unverified here:

  • Cloud NAT is definitely in use because archived NAT flow logs show stoxx-vm going through stoxx-nat, but exact NAT spend is not defensible without billing export.
  • Network egress, taxes, credits, and any future discounts are outside the evidence currently available in this project.
  • There is no billing export dataset, so the note cannot reconcile these estimates to invoice rows yet.

Data-Engineering Scenario Matrix

The current environment is small enough that scenario planning should be grounded in actual relevance, not just platform possibility.

ScenarioCurrent relevance to bq-wh-nbPrimary fixed driversPrimary variable driversDecision trigger
Lightweight API + scheduler + BigQueryLow todayNone significant yetQuery bytes, small serverless runtimeOnly becomes relevant if Cloud Run services and Scheduler are introduced.
Batch ingestion + Cloud Run Jobs + GCS + BigQueryHighMinimal fixed cost if VM dependencies are removedCloud Run seconds, GCS growth, BigQuery scansCurrent job history proves this path is viable and cheap at small scale.
Airflow-centric orchestrationHighstoxx-airflow VMLog growth, occasional job burstsThis is currently the largest single compute floor.
Streaming / event-driven ingestionLow todayCould add always-on consumers if introducedPub/Sub volume, consumer runtimePub/Sub topics and subscriptions are currently absent.
Warehouse-heavy analyticsLow todayBigQuery commitments if purchasedQuery bytes or slot-hoursNo reservations exist and current scan volume is tiny.

Recommendations / Decision Points

  • Do not buy BigQuery slots yet. Current 30-day analytical volume is only 0.000259954 TiB, and the project has no evidence of sustained warehouse pressure.
  • Treat VM rightsizing and off-hours shutdown as the biggest current lever. The fixed VM and disk floor dwarfs observed Cloud Run and BigQuery usage.
  • Do not remove NAT blindly. Live logs prove stoxx-vm currently uses stoxx-nat; remove or redesign only after confirming the traffic path can move to Cloud Run or Private Google Access.
  • Do not over-focus on GCS lifecycle yet. The bucket footprint is tiny today, though stoxx-sql-bucket could become material if it turns into a backup archive without lifecycle transitions.
  • Fix attribution before chargeback. The VM labels are useful, but the project still lacks billing export and BigQuery dataset labels, so cross-team TCO is not yet defensible.

Current product note: commitments are a second-stage optimization

Current Cloud Billing cost-optimization guidance keeps commitments and FinOps-hub-style savings review downstream of basic visibility. If billing export, attribution, and usage history are still missing, rightsizing always-on infrastructure is usually a safer first lever than commitment purchases or slot-style precommitment decisions.

Troubleshooting / Incident-Response Runbooks

TCO looks higher than the inventory-based baseline

  • Check for NAT, egress, tax, or logging charges that the baseline intentionally does not estimate.
  • Confirm whether any new always-on service was added outside the main VM and disk inventory.
  • Enable billing export before arguing over invoice deltas.

BigQuery suddenly becomes a serious driver

  • Rerun the JOBS_BY_PROJECT query by principal.
  • Rerun the referenced-table query from the monitoring note.
  • Only consider reservations or commitments after you have at least a sustained history of meaningful scan volume.

Cloud Run jobs stop looking cheap

  • Re-run job execution listings and recalculate with the same formula from this note.
  • Check for retries, concurrency changes, or runtime increases before changing architecture.

Month-end reconciliation is still impossible

  • Confirm again that no billing export dataset exists.
  • Enable standard usage export first.
  • Only then compare inventory-based estimates to billed cost by SKU, label, and service.

Quick Reference

QuestionArchived answer
What dominates current TCO?Running VMs and attached persistent disks.
Are BigQuery commitments present?No.
Are Compute Engine commitments present?No.
Is current Cloud Run usage material?No; observed runtime is financially negligible.
Is current BigQuery query volume material?No; it is still tiny and below the first free-tier breakpoint.
What is the biggest missing TCO input?Billing export, because it is needed for invoice-grade reconciliation.

Graph View

Backlinks