Per-token pricing is a distraction
You do not pay per token. You pay per user request, and each request is a sequence of choices you control: model tier, context length, retry policy, caching.
The three levers that matter
- Model tier routing. Send 80 percent of traffic to the cheapest model that passes eval; escalate the rest.
- Prompt caching. Providers now cache system prompts server-side. Structure yours to maximize the shared prefix.
- Structured output. JSON mode reduces retries. Retries are where budgets die.
Watch the FX line
Most providers bill in USD. Your CFO sees CAD. Build FX into your unit-economics model or you will misprice.
When to self-host
Under ten thousand daily requests, hosted wins. Over one million, self-hosting on Canadian infrastructure often wins on cost and always wins on data residency. In between, run the math quarterly.