How to meter and price every token type per customer from your AI gateway

If you resell model access, your invoice is only as good as your metering. A customer who sends long prompts and a customer who generates long answers cost you very different amounts, and a cache hit costs a fraction of a fresh input token. One blended "tokens" line on an invoice hides all of that.

Teams we talk to usually end up building the same pipeline by hand: scrape the gateway, join usage to a customer table, split it by token type, and push it to Stripe or Metronome on a cron. Then they maintain it.

With CostGraph you can meter every token type per customer, straight from your AI gateway, and price each one independently in your billing platform. No pipeline to build. This post walks through the whole flow, from the label on a request to the line on an invoice.

The flow end to end

1. Your gateway labels each request with a customer. A header, a request field, or the gateway's own consumer identity.
2. CostGraph reads the gateway's token metrics. The CostGraph operator in your cluster, or the agent on your VM, scrapes them. Each token type lands as its own row: input, output, cached input and cache write.
3. CostGraph prices the usage at your cost. Your negotiated rate, a catalog rate for the model, or the amortized cost of a model you host yourself.
4. Virtual tags split usage by customer and token type. One tag key per token type.
5. Billing exports send each customer's daily token count to a meter per token type. Stripe, Metronome, Orb, Lago, OpenMeter, Chargebee or Autumn applies your price and issues the invoice.

Steps 1 and 2 depend on your gateway. Everything after that is the same for all of them.

Before you start

You need:

• A CostGraph organization with an active subscription. Billing exports are part of the paid plan.
• An admin, owner or billing admin role to connect the gateway and the billing platform. Any member can create virtual tags.
• The CostGraph operator in the cluster that runs your gateway, installed with aiGatewayScraper.enabled: true, or the CostGraph agent on the virtual machine that runs it. See Connect an AI gateway.
• A secret key for your billing platform, such as a Stripe secret key from Developers > API keys. Each platform's key is listed in the billing exports docs.
• A CostGraph API key with the pricing:write scope, only if you publish your own model rates in step 3, or focus:write if you send usage yourself. Create it under Settings > Account > API keys at app.costgraph.ai. Only admins can create a key with pricing:write.
• A customer ID in your billing platform for each customer, such as cus_ABC123 in Stripe.

Step 1: attribute usage to your customers

Every supported gateway can carry a customer id on its token metrics. CostGraph reads it into the sub-account, so per-customer usage shows up anywhere you group cost by sub-account.

metricsServer:
    enabled: true
    port: 4001
proxy_config:
    litellm_settings:
        callbacks: ['prometheus']
        enable_end_user_cost_tracking_prometheus_only: true
        prometheus_exclude_labels: ['client_ip', 'user_agent', 'user_email']

curl https://litellm.example.com/chat/completions \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "user": "acme", "messages": [{"role": "user", "content": "Hello"}]}'

helm upgrade -i aieg oci://docker.io/envoyproxy/ai-gateway-helm \
  --version v1.1.0 -n envoy-ai-gateway-system --reuse-values \
  --set controller.metricsRequestHeaderAttributes="x-costgraph-customer-id:customer.id"

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
    name: costgraph-customer-attribution
    namespace: my-gateway-namespace
spec:
    targetRefs:
        - group: gateway.networking.k8s.io
          kind: Gateway
          name: my-gateway
    frontend:
        metrics:
            attributes:
                add:
                    - name: customer
                      expression: request.headers["x-costgraph-customer-id"]

kubectl annotate deployment/llama-serving costgraph.ai/sub-account=acme

Step 2: connect the gateway to CostGraph

Open Settings > Integrations, pick your gateway, and choose where it runs: a Kubernetes cluster, where the CostGraph operator scrapes it, or a virtual machine, where the CostGraph agent does. That is the whole connection. There are no credentials, because CostGraph reads the telemetry your gateway already emits.

Each token type arrives as its own row, with a meter named <model>/<token type>. For example, gpt-4o/output is the output tokens of gpt-4o.

Step 3: price the usage at your cost

CostGraph prices every row, so you see your margin next to what you bill. Rates resolve in order:

1. Your own rate, published once and pinned to the gateway with a costgraph.ai/pricing-id.model label.
2. A catalog rate for the same model.
3. The amortized cost of a model you host yourself. This is what the serving namespace cost that day, divided by the tokens it served.

You do not have to publish anything. Without a rate of your own, CostGraph uses the catalog rate or the amortized cost.

Publish your own rate

Rates are published through the CostGraph Pricing API. Publish one entry per token type with the pricing:write API key from Before you start. Prices are per million tokens. Cached input is cache_read here.

curl -X POST https://pricing.baselinehq.cloud/v1/marketplace/models \
  -H "X-API-KEY: $COSTGRAPH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "entries": [
      {"model": "meta/llama-3.3-70b", "token_bucket": "input",       "unit_price_per_mtok": 0.60, "currency": "USD"},
      {"model": "meta/llama-3.3-70b", "token_bucket": "output",      "unit_price_per_mtok": 0.80, "currency": "USD"},
      {"model": "meta/llama-3.3-70b", "token_bucket": "cache_read",  "unit_price_per_mtok": 0.15, "currency": "USD"},
      {"model": "meta/llama-3.3-70b", "token_bucket": "cache_write", "unit_price_per_mtok": 0.75, "currency": "USD"}
    ]
  }'

The response returns an id for each entry. Label the Deployment or namespace of the gateway CostGraph scrapes with the id of the input entry. CostGraph finds the other token types of the same model from it. A pin prices every model on that gateway connection at the pinned model's rates, so pin a gateway that serves one model.

Set INPUT_ENTRY_ID to the id of the input entry, then label the gateway:

kubectl label deployment/my-gateway costgraph.ai/pricing-id.model="$INPUT_ENTRY_ID"

Pins apply to gateways in Kubernetes. Usage from a gateway on a virtual machine, or usage you push, is priced at the catalog rate. Publishing the same model and token type again with a new price replaces the old rate. The full request is in the publish model prices reference.

See your rates

Every rate you publish shows up under Settings > Organization > Prices. The AI models tab lists each model with its input and output price per million tokens. Next to each rate it shows the cheapest matching public price, so you can see whether your negotiated rate is still a good deal. Admins can delete a rate there.

This is your cost. What you charge your customers is set in your billing platform in step 5.

Step 4: split usage by customer and token type

Attribution uses virtual tags, the same rules that already assign cloud spend to teams and customers. Create one virtual tag key per token type, named with your billing platform's prefix, for example stripe_tokens_input. On each key, add one rule per customer that matches the customer and the token type and assigns that customer's id in your billing platform:

This is the rule for Acme's output tokens, built on the stripe_tokens_output key:

We ran this on our own test cluster, which runs agentgateway with two test customers, acme and globex. Between September 11 and October 19, 2026, acme sent 67.46 million input tokens and 67.46 million output tokens. The input tokens cost $13.50 and the output tokens $54.01. The token counts are almost identical, but output cost four times as much. A single blended token price hides exactly that.

The two rules split acme's 78 daily rows evenly, 39 each, with none left over and none counted twice.

Select View claims on any rule to see the rows and spend it matches before you rely on it.

Step 5: export to your billing platform and price each token type

This is the same nightly export we launched with Stripe, pointed at token counts instead of cloud spend. In your billing platform, create one meter per token type and give each its own price. Then bind each tag key to its meter in quantity mode, with the unit tokens. Every night, CostGraph sends each customer's daily token count to each meter.

Those prices are an example. Tiers, minimums, commitments and per-customer discounts all live on the meter's price, so you can reprice one token type without touching CostGraph or your gateway.

Prefer to charge a markup on what the tokens cost you? Bind the same tag keys in cost mode instead. CostGraph sends each token type's cost in cents, and your platform applies the markup.

Get started

Token metering uses parts of CostGraph you may already have:

• Connect an AI gateway
• Create an API key and read how API keys authenticate
• Publish model prices and pin them with pricing IDs
• Create virtual tags
• Set up billing exports

If you want help wiring it up for your gateway and billing platform, get in touch and we will walk through it with you.

FAQ