It was a Tuesday. I opened my email between two meetings and there it was. An invoice from OpenAI. $47,000.
I stared at it for a good thirty seconds before my brain accepted the number was real.
The month before, we'd paid around $6,000. I expected maybe $8,000 — we'd launched a new summarization feature that was getting traction. But $47,000? That's not growth. That's something broken.
I spent the next four hours trying to figure out what happened. And that's when I realized the scariest part wasn't the bill itself. It was that I had absolutely no way to trace it.
What went wrong
Here's what I pieced together after digging through logs, Slack threads, and git blame.
Three weeks earlier, a developer on my team had been testing a new prompt chain for our document analysis feature. The chain called GPT-4 (this was back when GPT-4 was $0.03/1K output tokens — expensive). Each call processed a 15-page PDF and generated a structured summary.
The test worked. He pushed it to staging. Someone else merged it to production without realizing the batch processing loop had no rate limit and no cap. It ran against our entire document backlog — 12,000 documents — over a weekend.
Nobody got an alert. Nobody saw a dashboard spike. Because we didn't have alerts. We didn't have a cost dashboard. We had OpenAI's billing page, which updates 24-48 hours late, and a monthly invoice that arrives after the damage is done.
By Monday morning, the batch job had finished. It had consumed over 800 million tokens in 72 hours.
What I should have seen
Looking back, there were three moments where a single check would have saved us $41,000.
A hard spending cap. If we'd set a monthly limit of $10,000 — even a generous one — the system would have blocked API calls after that threshold. The batch job would have processed maybe 2,500 documents instead of 12,000. We would have noticed Monday morning with a $10,000 bill and fixed it before lunch.
Per-feature cost tagging. Our chatbot feature and our document analysis feature both used GPT-4. On the OpenAI invoice, it was all one number. If every API call had been tagged with which feature triggered it, I would have seen "document analysis: $38,000" and immediately known where to look.
A real-time alert at 50% of budget. We budgeted $8,000 for the month. An alert at $4,000 — hitting on Saturday morning — would have given us two full days to investigate before the damage tripled.
None of these are complicated. A budget cap is a counter in Redis. A tag is an HTTP header. An alert is a threshold check after each API call. But when you're building product features and shipping fast, cost monitoring is always the thing you'll set up "next sprint."
Next sprint never comes until the invoice does.
What I built after
I couldn't find a tool that did what I needed. Helicone was $79/month and focused on observability — traces, logs, evaluations. I didn't need to debug my prompts. I needed to know how much money each feature was burning before the month ended.
So I built Tokonomics.
It sits as a proxy between your app and the LLM provider. Every API call gets logged with the model, token count, cost, and whatever tags you attach — feature name, customer ID, environment. You set budget alerts at any threshold. You set hard caps that physically block API calls when the budget runs out.
If I'd had it that weekend, the batch job would have hit the $10,000 cap on Saturday at 2am, stopped, and waited for someone to either raise the limit or investigate. Total cost: $10,000 instead of $47,000.
The invoice that broke my budget is the reason the product exists.
The number that still bothers me
It's not the $47,000. We survived that. What bothers me is this: when I talked to other founders about it, almost every single one had a similar story. Maybe not $47,000 — but $3,000 here, $8,000 there, a month where the AI bill doubled and nobody could explain why.
A 2026 a16z survey found that 71% of companies using LLM APIs don't track spending at the individual call level (a16z, 2026). They get one invoice, once a month, with one number.
If that's you right now — set up a cap. Any cap. Even a generous one. It takes ten minutes and it's the difference between a surprise and a disaster.
Tokonomics tracks every LLM API call by model, feature, and customer — with budget alerts and hard spending caps. Free plan available.