21/09/2026
Kubernetes inference cost tracking breaks on shared components.
Per-namespace allocation is where most platform teams stop. It is enough for chargeback and not enough for a product decision, because it cannot tell you which model, tenant, or feature is economically responsible for the spend.
Model level allocation has to account for four things: GPU memory reserved for weights whether or not the model is serving, active compute during inference, the shared gateway, and KV cache storage. OpenCost's integration with llm-d now tracks this at the model level in Kubernetes.
The reason this matters is not reporting. GPU cost per hour cannot tell you whether an AI feature is economical. Cost per successful inference, attributed to a feature, can.
Decide your allocation rules before you optimise capacity. Optimising a workload you cannot attribute is how teams cut GPU spend and lose a product line.
Can you name the highest cost model running in your production cluster right now?