Engineering4 min read
ByEmmanuel Bakare
Splitting instance prices with linear programming
How CostGraph estimates CPU, memory, and GPU costs from each machine family’s price list while keeping the total unchanged.
ChatGPTA cloud machine has one price, but CostGraph needs separate CPU, memory, and GPU costs. We now estimate those costs from each provider's price list while keeping the total unchanged.
The problem
An AWS r7i.large costs $0.1323 an hour on the EC2 price list. For that you get 2 vCPU and 16 GiB of memory. AWS does not say how much of the $0.1323 is the CPU and how much is the memory, because you cannot buy them on their own.
When CostGraph tells you what a pod costs, or how much a smaller machine would save, it has to price CPU, memory, and GPU separately.
The old way
Google Cloud publishes separate CPU and memory prices and GPU prices. For everyone else, we assumed a vCPU cost 7.5 times as much as a GiB of memory.
Published splits use provider rates. Our estimates used the 7.5 rule, which gave these two AWS machines different CPU costs despite both having 2 vCPU. The totals stayed correct, but the CPU estimates were inconsistent.
Our approach lets the list price decide
Since a provider such as AWS does not directly tell you the price of one vCPU, we use the price differences between machines. If adding 8 GiB of memory adds $0.03, that helps us estimate what memory costs. We compare prices across a family of machines and find the CPU and memory rates that fit them best.
Grouping machines into families
We only compare machines built on the same hardware. AWS's naming guide helps identify them. In c7i, the c means compute optimized, 7 is the generation, and i is Intel. c7i, m7i, and r7i use the same Intel chips with 2, 4, and 8 GiB per vCPU, so we treat them as one family. Graviton-based c7g and AMD-based c7a use different chips, so each gets its own family.
Finding the best fit
The three .large machines in the 7i family all have 2 vCPU. Only their memory and price differ. On a chart of price against memory, a line's starting value estimates the CPU cost. Its slope estimates the cost of each GiB of memory.
No straight line hits all three prices exactly, so we look for the one with the smallest error. The following demo compares three candidate lines.
- try 1through c7i and m7i6.4% off
- try 2through m7i and r7i4.7% off
- try 3through c7i and r7i2.8% off
We express this as a linear programming problem. Our solver uses the simplex method, which moves between candidate solutions to find the lowest error. Gradient descent instead takes small steps downhill.
We look for a price per vCPU and a price per GiB. Neither can be below zero. For each machine, we compare the predicted total with its listed price and measure the error as a percentage. This gives cheap and expensive machines equal weight. We choose the pair with the smallest total error.
In the following formula, is the price of one vCPU and is the price of one GiB.
For the 7i family, the fitted ratio is 10.44 GiB of memory per vCPU, compared with the old ratio of 7.5. The following demo shows how that changes each machine's split.
With the new ratio, c7i and r7i now agree on what a vCPU costs.
Adding the GPU
GPUs work the same way. The five smallest AWS g5 sizes, from g5.xlarge to g5.16xlarge, each have one A10G GPU. A g5.16xlarge has 16 times the vCPUs and memory of a g5.xlarge. We keep the CPU to memory ratio we already learned and fit the GPU price against each machine's full price.
It is the same chart as before. On the 7i chart, every machine had 2 vCPU, so the line's starting value was the CPU cost. Here every machine has one GPU, so the line's starting value, where CPU and memory are zero, is the GPU cost.
Each extra 4 vCPU and 16 GiB adds $0.206. Following the line back to zero CPU and memory gives $1.006 - $0.206 = $0.800 an hour per A10G. The line hits all five prices exactly, so their price differences come from CPU and memory.
The bigger sizes check the answer. g5.12xlarge and g5.24xlarge have 4 GPUs, and g5.48xlarge has 8. Price them as $0.800 per GPU plus the same CPU and memory prices, and you get $5.672, $8.144, and $16.288 an hour, exactly what AWS charges.
How the parts add up
The fits produce two values: the CPU to memory ratio and the price per GPU. Each machine's CPU and memory costs then come from its own listed price:
- Subtract the GPU cost from the machine's price. The rest is the cost of its CPU and memory.
- Split that cost between CPU and memory using the ratio.
The parts therefore always add up to the listed price. For a g5.xlarge:
When a family's prices follow the line exactly, like g5, every size gets the same CPU and memory prices. When they do not, the GPU price and the ratio stay fixed and each machine's CPU and memory prices absorb its own difference. m7i.large is 2.8% off the 7i line, so its vCPU price is $0.0365 an hour, compared with $0.0375 for c7i.large and r7i.large.
All together
We repeat this process whenever a provider's prices refresh.
- 01group
Group machines with the same generation and silicon, regardless of memory per vCPU.
c7i + m7i + r7i -> 7i - 02learn CPU and memory costs
Linear programming finds the vCPU and GiB costs that best explain the family's prices.
7i → 1 vCPU = 10.44 GiBif it cannot be learnedprovider's median ratio, then the ratio Google Cloud publishes - 03learn the GPU
With that ratio held fixed, the same approach finds one rate per GPU model across its family.
g5 → $0.800 per A10Gif it cannot be learnedwhat is left after CPU and memory - 04split each machine
Each machine is split on its own price, so the parts always add back to it.
cpu + memory + gpu = price
Sometimes there is nothing to compare. A family with only one size has no steps to learn from. In that case we borrow the usual ratio from the same provider, or from Google Cloud if the provider has none.
Discounts keep the same split. A spot or reserved machine is split the same way as its full-price version, just with every part scaled down.
Across every provider
The same process runs for every provider we support, each time its prices refresh. What comes out depends on how each one sets its prices.
- Google Cloud publishes CPU, memory, and GPU prices, so we use them directly. They also serve as a reference for checking our estimates and as the final fallback for other providers.
- For AWS and Azure, we learn a ratio for almost every family. Their ratios are around 10, compared with the old 7.5.
- Providers with bundled extras or tiered pricing, such as DigitalOcean, use Google Cloud's ratio of 8.79.
- GPU providers such as RunPod and Vast list a price per GPU-hour, which we use as the GPU cost.
- For the largest AWS and Azure training machines, GPU, CPU, and memory capacity grow together. That leaves too little variation to isolate a GPU price, so we estimate its share.
Try it on your machines
Connect a cloud account and open a node in CostGraph to see its CPU, memory, and GPU cost breakdown. Get started for free or book a demo with us.
Frequently asked questions
No. The split only decides how a machine's price is shared between its CPU, memory, and GPU. Every machine's parts still add up to the price your provider charges.
The fixed ratio gives the AWS c7i.large and r7i.large different CPU costs despite having the same 2 vCPU. Fitting a ratio to the family's prices gives us an estimate based on that hardware's pricing.
Every cloud and GPU provider CostGraph supports. The split is worked out again each time a provider's prices refresh, so it follows their price changes.
It uses the usual ratio from the same provider. If none is available, it uses a ratio derived from Google Cloud's published rates.
The same way as the full-price machine, with every part scaled down by the discount.