Compare GPU infrastructure costs before you deploy

What started as a “we probably could” internally is now a feature. Starting today, we are taking another step towards making cost visible across your organization, including where your infrastructure changes are reviewed.

Alongside our Infracost integration, we are releasing a GitHub Action to estimate the cost of your Terraform plans and help you find cheaper instances across clouds.

A big reason we built this is GPUs. If you have tried to find somewhere to run a model, you know how quickly the search becomes a collection of pricing pages, instance names, and regions. The GPU you want might be offered by several providers, bundled with very different amounts of CPU and memory. Working out which option makes sense for your workload takes more than comparing the first hourly price you find.

And once you have picked an instance, that choice tends to follow you into the Terraform configuration. We wanted to make it easier to revisit the decision while a change is still being discussed.

Why look beyond your usual cloud?

If you serve models, the cost of the hardware feeds directly into what it costs to serve your customers. You might be adding a new model, moving an inference workload, or looking for another place to run it because your usual region cannot give you the GPUs you need.

In each case, there is more to compare than the name of the accelerator. GPU count, memory, the rest of the instance, and location all matter. You need enough memory for the model, a location that works for your users, and capacity you can actually provision.

Our last release focused on understanding the GPUs you already run. This release brings cost into the decision about what to run next. CostGraph can show instance options across providers alongside your Terraform estimate, giving you a starting point for that search without leaving the pull request.

These are pricing comparisons, not a live inventory of available GPUs. You still need to confirm capacity, quotas, and workload compatibility with the provider before moving.

What we shipped

Bring your rates and more providers into the estimate

If your Infracost estimates use list prices, they may miss the effect of commitment discounts or negotiated rates on what you actually pay. Infracost supports custom price books, but those rates need to be configured. And if you run infrastructure outside AWS, Azure, and Google Cloud, you need pricing coverage for those providers too.

Point Infracost's CLI at CostGraph and keep your existing workflow. For AWS, CostGraph adjusts estimates using the effective rates on your bill over the last 30 days, including discounts reflected in that billing data. SKUs you have not used fall back to list price; Azure and Google Cloud currently use list prices.

CostGraph also supports custom pricing for your own compute and storage offerings. Alongside the Infracost integration, our GitHub Action adds Terraform pricing for supported resources from providers such as Hetzner, DigitalOcean, and Vultr. That gives you a broader view of what the infrastructure in your plan could cost.

Compare instance options in the pull request

The GitHub Action reads your Terraform plan and posts a cost comment, updating the same comment when you push another change. It combines Infracost estimates with CostGraph's coverage of providers such as Hetzner, DigitalOcean, and Vultr.

The Comparable instances table puts instance sizes, locations, monthly prices, and potential savings next to each other. It includes GPU details where the pricing data provides them. Review the returned locations as well as the hardware: some catalog offers use a global region label, and a cheaper option may be in a different geography.

For a GPU workload, the accelerator type and count help you narrow the options. You can then test whether the cheaper candidate delivers the throughput your model needs.

Keep the estimate with the change

The result also appears in the workflow's job summary. You can save the estimate as an artifact in Infracost JSON format for a later check or review, and combine several Terraform plans into one comment.

Try it on a pull request

We have a small demo repository and an example pull request you can inspect. The repository only generates plans; it uses deliberately invalid AWS credentials and must not be applied. For the GPU comparison below, we ran the same demo locally with these values in terraform.tfvars:

instance_type  = "g4dn.xlarge"
instance_count = 1

The AWS G4dn family uses NVIDIA T4 GPUs. Generating this plan does not provision a GPU.

1. Connect Infracost and create a CostGraph key

In CostGraph, open Integrations → Infracost and connect your Infracost API key. You can find the key your local CLI uses with infracost configure get api_key.

Next, create a CostGraph API key with pricing read access in account settings. In your GitHub repository, open Settings → Secrets and variables → Actions and save it as COSTGRAPH_API_KEY.

The workflow uses that CostGraph key. Your Infracost key stays in CostGraph.

2. Pass the Terraform plan to the action

In an existing Terraform workflow, export the saved plan as JSON and add the cost step:

- name: Export the plan
  run: terraform show -json tfplan > plan.json

- name: Review infrastructure cost
  uses: baselinehq/costgraph-action/[email protected]
  with:
      api-key: ${{ secrets.COSTGRAPH_API_KEY }}
      plan-path: plan.json
      artifact-name: cost-estimate

Here, tfplan is the file produced by terraform plan -out=tfplan. Set terraform_wrapper: false in your hashicorp/setup-terraform step so the JSON output stays usable, and give the job permission to post its comment:

permissions:
    contents: read
    pull-requests: write

The complete workflow includes checkout, Terraform setup, and planning. Your plan step may need cloud credentials; the cost action itself only needs the plan and your CostGraph key. Run this secret-backed workflow for trusted branches in your repository; fork pull requests do not receive the secret.

3. Read the alternatives before choosing an instance

Open a pull request that changes an instance type or count. Once the workflow runs, inspect the estimate and the Comparable instances section.

This is the alternatives output from our local GPU plan, rendered from the same CLI report format the action uses. On October 6, CostGraph priced the AWS g4dn.xlarge in us-east-1 at $383.98 a month. The returned options included Vast's p100-2x at $41.10, Clore's rtxpro2000-1x at $54.45, and Hyperstack's n3-RTX-A4000x1 at $109.50.

That puts the estimated monthly instance prices 89%, 86%, and 71% lower, respectively.

Those options use different GPUs and locations. A pair of P100s is not the same hardware as a T4, and a lower instance price does not guarantee a lower cost per request. Use the comparison to find candidates, then check your model's memory requirements, benchmark throughput, and confirm capacity with the provider. These are per-instance pricing estimates, separate from the total for the proposed plan.

The proposed monthly total and a before/after change are different things. Previous cost and the change are available when the estimate includes previous costs; the action does not take a separate base-branch plan as an input.

Already using Infracost locally?

You can try the integration without a GitHub workflow. With Infracost CLI v0.10 installed and your CostGraph key exported as COSTGRAPH_API_KEY, run:

export INFRACOST_PRICING_API_ENDPOINT=https://api.costgraph.ai/api/v1/tenant/infracost
export INFRACOST_API_KEY="$COSTGRAPH_API_KEY"
infracost breakdown --path plan.json

The integration guide has the settings for CLI v2 and the full provider coverage.

Give it a try

Finding somewhere to run your GPUs already involves enough tradeoffs. Comparing what those options cost should be an easy part of the decision, and the people reviewing the infrastructure should be able to see it too.

Add the GitHub Action or connect Infracost. We would love to hear what you find when you compare the instances you use today.

Frequently Asked Questions