Amazon SageMaker
Real-time endpoints and notebook instances.
AI Infrastructure and AI Economics
Every AI token and dollar, traced to the team that answers for it
Varcio AI cost management covers the infrastructure behind your models and the money you spend on them: findings and one-click actions for SageMaker, Bedrock, Vertex AI and Azure OpenAI, plus a ledger that traces AI spend from any cloud or tool to the team, person, service and business unit answerable for it.
4AI services with findings and actions: SageMaker, Bedrock, Vertex AI and Azure OpenAI
4Multi-cloud AI detectors: context, routing, reasoning tokens and retry storms
6AI Economics screens, from accountability to controls
Per 1MCost per million tokens and per request, per feature and customer
AI Infrastructure
Findings and one-click actions for the services behind your models
AI workloads are not served by general compute detectors: GPUs idle differently, endpoints are provisioned differently and inference cost scales with tokens, not instance hours.
- Idle endpointFound by a detector
- CostedSavings, confidence, effort
- ApprovedPolicy and approval
- Action runsAudited
Amazon Bedrock
Provisioned throughput and model invocation visibility.
Google Vertex AI and Gemini Enterprise
Prediction endpoints and Workbench instances.
Azure OpenAI, AI Foundry and Azure ML
Model deployments and managed compute instances.
Other GPU and ML workloads
A spend-based signal for GPU workloads outside the services above.
GPU intelligence
Per-container GPU cost attribution and utilisation efficiency, on the Business plan and above.
An AI service taxonomy classifies spend across training, inference, endpoint hosting and model API consumption, so the bill can be reasoned about by workload type rather than by SKU. Findings carry the same savings, confidence, effort and risk scoring as every other detector output, and each supported provider has its own remediation action module.
Idle endpoints and over-provisioned GPU instances are typically the largest and safest early wins. A model pricing catalogue and a side-by-side model cost comparison let you test whether a cheaper model meets the quality bar for high-volume, low-complexity calls.
What you can do
- Work findings with an accept and dismiss workflow
- Detect AI spend anomalies specific to AI workload shapes
- Set AI cost guardrails per workspace
- Compare model prices side by side
- Ingest AI cost data from sources outside the cloud bill
AI Economics
Accountability for AI spend: who spent it, on what, and why it moved
AI spend is usually the least attributed line on the bill. This makes the share nobody answers for impossible to miss.
Per 1M tokens$2.40By model
Per request$0.004By feature
Per customer$18By account
Accountability
Who spent it: teams, business units, services, people, tools and cloud accounts, in dollars and tokens, with what changed and who answers.
Models and providers
Where the money flows from provider to model to team, and what each model costs per million tokens against its list price.
Why it moved
A change in AI cost split exactly into more requests, longer requests, a shift between models and a change in rate.
Efficiency
Cache hit, output share, failed calls and price against list, by team and model, and when the usage happens.
Coding tools and seats
What developers did with coding assistants beside what the seats are worth, and whether the seats you pay for are used.
Controls
AI budgets, the check your code can ask before calling a model, who-is-spending rules, where usage comes from, and how people are shown.
How it stays honest
Spend with no owner is a row, not a rounding error
The rules that hold on every screen.
Tokensfive classes
- Input
- Output
- Cache read
- Cache write
- Reasoning
| Rule | Consequence |
|---|---|
| Spend with no owner is a row | Every breakdown carries a none row, with its dollars, so the share nobody answers for is always on screen. |
| Tokens and dollars side by side | Tokens are split by class (input, output, cache read, cache write, reasoning). Dollars are the ledger’s cost. They are never blended. |
| People shown as you chose | Named or pseudonymised, with the pseudonym applied on the server. |
| Business units are derived | They are never stored on a row. |
| Like-for-like windows | The comparison window is the same number of days directly before. |
| No guessing in unit economics | A usage row with no customer is reported as not attributed rather than divided, and a margin appears only where revenue for that customer or feature exists. |
Scroll sideways to see the full table.
Attribution
Write the mapping once, and every past and future row carries it
Provider usage tells you which key spent the money. Only you know what the key is for.
- Usage rowsPast and future
- MappingWritten once
- OwnerTeam, service, feature
Tie an OpenAI project, an Anthropic workspace, an API key, a person or a cloud account to a feature, a team, a service and a tool. A key’s own mapping wins over its project’s, and usage nobody has mapped stays unattributed and is listed with its cost.
Seat-priced AI tools have no API reporting their cost. Enter each one once, with plan, monthly cost, seats and the teams holding them, and the register prorates the cost to the period and splits it by seats held, reporting unassigned seats as unassigned rather than spreading them.
Where AI usage comes from
- Direct provider connections for OpenAI, Anthropic and others, with an encrypted admin key
- Cloud-hosted AI on SageMaker, Bedrock, Vertex AI and Azure OpenAI
- OpenTelemetry GenAI spans, so existing instrumentation can feed the ledger without a Varcio SDK
- A direct ingestion endpoint for sources outside the standard billing path
Detection
Detectors that look at how you call models
Alongside idle endpoints and GPUs, four multi-cloud detectors look at AI cost patterns.
- Context reductionSend less with each call
- Model routingRoute to a cheaper model
- Reasoning tokensOverhead on a call
- Retry stormsRetries that inflate spend
Context reduction
Opportunities to send less context with each call.
Model routing
Opportunities to route calls to a cheaper model.
Reasoning-token overhead
Reasoning tokens that add cost to a call.
Retry storms
Repeated retries of failed calls that inflate spend.
FAQ
Questions, answered
What does Varcio AI cost management cover?
AI Infrastructure covers findings and one-click actions for Amazon SageMaker, Amazon Bedrock, Google Vertex AI and Azure OpenAI or AI Foundry. AI Economics traces every AI token and dollar, on any cloud or tool, to the team, person, service and business unit answerable for it, and explains why it moved.
Can it track spend with providers outside my cloud bill?
Yes. Connect OpenAI, Anthropic and other providers with an encrypted admin key and usage is pulled on a schedule into the same ledger as your cloud-hosted AI spend. OpenTelemetry GenAI spans and a direct ingestion endpoint are also supported.
How does Varcio calculate cost per request and per million tokens?
From the ledger. Nothing is spread or guessed: a usage row with no customer is reported as not attributed, a margin appears only where revenue for that customer or feature exists, and cost from a billing export, where token counts are approximate, is labelled so per-token figures built on it read as estimates.
Does it cover GPUs?
Yes. Findings cover over-provisioned and idle GPU instances and endpoints, and GPU intelligence adds per-container GPU cost attribution and utilisation efficiency on Business and above.
Which plans include it?
AI Infrastructure and AI Economics are available on every plan. GPU intelligence is available on Business and above.
Keep reading
Related reading
FinOps for AI: How to Control LLM Token Spend Before It Controls Your Budget
98% of FinOps teams now manage AI spend, up from 31% two years ago. Here is a practical playbook for tracking token costs, cutting LLM inference bills, and tying AI spend to business value.
Read the articleIdle GPUs Are the New Cloud Waste: Why Kubernetes GPU Utilization Averages 5%
Cast AI analyzed about 23,000 Kubernetes clusters and found average GPU utilization of 5%. Here is what that means for your bill and the practical steps that bring it down.
Read the articleAzure OpenAI vs. AWS Bedrock vs. Google Gemini Enterprise: An Enterprise Buyer's Guide
Choosing a foundation model provider is now an infrastructure decision, not just a model-quality decision. Here is how Azure OpenAI, AWS Bedrock, and Google Gemini Enterprise actually compare.
Read the articleFind out who owns your AI bill
Connect your clouds and AI providers and see accountable AI spend. Free trial, no credit card required.