A founder called us three months after launch with a complaint about our invoice. Not the build invoice, which he had approved line by line. The other one: the monthly model API bill plus the part-time contractor clearing the review queue, a sum nobody had quoted him because nobody had asked. His support-drafting assistant worked well, at roughly four times his budgeted running cost. The build was a fixed price. The running was open-ended, and open-ended was the part that mattered.
That conversation repeats on nearly every LLM project we scope. Vendors quote the build because the build is quotable. The running cost lives in five lines that rarely appear in proposals: tokens, retrieval infrastructure, evaluation upkeep, review labor, and the upgrade tax. All five belong in one monthly budget. The table below covers three usage tiers with planning figures from systems we run for clients.
What the meter actually measures
Tokens are the only line everyone expects, and even that line surprises people. Cost per run is not one model call. A classification step makes one call, but an agent-style loop makes several, with each step re-sending the transcript with tool results folded back in. Retrieval adds a second charge on top: every run pays for a search round-trip against your document index and a longer prompt stuffed with whatever the search returned. One of our client flows averages 9,000 input tokens per item processed, of which the actual customer email is under 400 words. The rest is instructions and retrieved context. That ratio is normal, and it is where the money goes.
Retrieval infrastructure is the line buyers forget entirely. Somewhere your documents live in searchable form: a hosted vector index, a Postgres with pgvector, an embedding pipeline that re-processes files when they change. At pilot scale this rides on free tiers and existing servers, effectively zero. Past a few hundred thousand chunks, or where access permissions must be enforced per user, it becomes a real system with hosting fees and someone responsible for re-indexing. Budget $0 to $50 a month at pilot scale and $200 to $800 where search is load-bearing. The failure mode is not the fee. It is stale search feeding the model last quarter's price list, which no token budget can fix.
Model choice multiplies everything above. As a rough planning ratio we use daily: a flagship-class model costs an order of magnitude more per token than a small model on the same task. Most workflow steps do not need the flagship. Classification and extraction from clean layouts run fine on small models in our experience, with the large model reserved for ambiguous cases and customer-visible text. Getting this routing wrong is the most expensive unforced error in LLM operations: flagship pricing on every step of a high-volume flow.
The labor bill next to the API bill
Every model step in production needs weekly human attention, and human attention is priced in hours, not tokens.
The review queue is the largest labor line. Low-confidence output goes somewhere for a person to check before it counts: a screen or a shared channel. Staffing it takes 15 to 30 minutes a day at pilot volume and grows with item count and error rate. Skip it and errors reach customers. Staff it with someone who approves everything unread and you pay the hours for zero protection. The wholesaler chatbot we inherited had 400 unread review items after six weeks without an owner, while staff had quietly returned to phone and email.
Prompt maintenance is the second line. Prompts are not write-once artifacts. Accuracy slips as input mix shifts, as retrieved documents change, and as the provider alters the model underneath you. Someone reads the misses each week, adjusts instructions, and confirms the fix against the test set. Budget a few hours a month per workflow at pilot scale, more where output faces customers.
Evaluation upkeep is the third line and the one teams resist until their first silent regression. A test set of 50 to 200 real inputs with known-good outputs, re-run after every prompt change and every model upgrade. Building it costs a day or two, keeping it current a few hours a month. It is also the asset that makes everything else cheap: with it, upgrades are a routine; without it, every change is a gamble. Our AI automation engagements hand this setup to clients as a deliverable, because a model step without a test set is a liability with an API key.
The upgrade tax
Model providers ship new versions on their schedule, not yours. Old versions get deprecated and behavior shifts in ways no changelog fully captures. A prompt tuned to perfection on one version can score measurably worse on the next. We have watched a classification step drop from 97 to 91 percent agreement with the test set across a single provider upgrade, with no change on our side at all.
Each upgrade therefore means the same routine: re-run the evaluation set, read the diffs, re-tune where scores dropped, and re-validate before switching traffic. Budget roughly a day of work per upgrade per workflow, two to four times a year depending on the provider. Where output affects customers, add a shadow period with the new version answering in parallel while humans compare. None of this is difficult. All of it is unbudgeted in the proposals we review. Treat it as a dependency someone else edits quarterly.
The monthly budget at three tiers
The table below uses planning ranges from systems we operate, labor valued at internal cost. Your numbers will differ, but the shape will not: the API line scales with volume while the labor lines arrive in steps, and labor exceeds tokens at every tier below very high volume.
| Monthly line | Pilot: one workflow, hundreds of items a week | Growing: two to three workflows, thousands of items a day | Scale: core process, 100k runs plus a month | | Model API spend | $40 to $120 | $300 to $900 | $1,500 to $5,000 | | Retrieval and hosting | $0 to $50 | $50 to $200 | $200 to $800 | | Review labor | 8 to 16 hours | 40 to 80 hours | Half a position and up | | Eval and upgrade care | 2 to 4 hours | 1 to 2 days | 3 to 5 days | | Indicative all-in total | $400 to $1,200 | $1,500 to $4,000 | $5,000 to $15,000 |
Two rows surprise buyers most. The first is review labor at the pilot tier: 8 to 16 hours a month exceeds the API spend several times over, and it is the line most likely to be zero in a vendor quote. The second is the step from Growing to Scale, where review becomes headcount rather than a side task. That transition deserves its own hiring decision, not a quiet accumulation of queue-clearing Fridays.
Read the table as a ceiling, not a floor, if your process tolerates plain automation for most steps. Every step that moves from model to rules deletes its token cost, its review minutes, and its share of upgrade care permanently. Our comparison of agents against plain workflows draws this boundary in detail: variance that rules cannot capture is the only thing a model step is strictly better at. The cheapest LLM budget is a smaller footprint. The cheapest line is the workflow that never needed a model.
Four levers that move the number
When the API line still hurts after scoping tightly, four levers work in our experience, ordered by effort.
Cache repeated context. System instructions and product catalogs get re-sent with every call by default. Providers now cache that repeated prefix so you pay full price once and a fraction after. On workloads with heavy shared context this roughly halves token spend for an afternoon of integration work. Do this first.
Route by difficulty. Send every item through the small cheap model, and escalate only low-confidence items to the flagship. In our builds the flagship sees 10 to 20 percent of items while quality holds, cutting blended per-item cost by more than half.
Cap retrieval depth. Each additional retrieved chunk lengthens every prompt in the loop. Measure accuracy against chunks retrieved: most workflows we tune stop improving after three to five chunks, and everything beyond that is pure cost. Set the cap where the curve flattens, not where it feels safe.
Batch offline work. Anything that does not need an answer in seconds belongs in batch APIs at roughly half the interactive price. Nightly classifications and report drafts are typical candidates. Splitting a workflow into an instant path and a nightly path is an hour of design and the easiest discount available.
None of these levers replace the labor lines. They shrink the API line, already the smaller half at most tiers: tokens are optimizable, attention is structural.
Budget the running before you approve the build
The sequence we recommend follows from the table. Price the monthly budget before signing the build, using your item counts and the ranges above. If the all-in total alarms you, shrink the model footprint first with fewer steps on models and strict retrieval caps. If it still alarms you, the process may belong in code. Our breakdown of MVP and product costs makes the same point from the build side: the cheapest system to run is the one with the fewest moving parts that still does the job.
Keep one scenario in the spreadsheet: prices doubling or volume tripling, both of which hit clients of ours within a year. Tightly scoped systems survived; flagship-everywhere did not.
If you have target volumes in mind, that is enough for a real number. Ours runs from Berlin with senior engineers behind it, and we put the monthly budget in writing before kickoff, labor lines included. Send us the process and the volumes and we will return the table with your figures in it.
Frequently asked questions
How much does it cost per month to run an LLM feature?
For one workflow at a few hundred items a week, plan on $40 to $120 in model API spend plus $0 to $50 in infrastructure, with 8 to 16 hours a month of human review and maintenance on top. All-in monthly totals land around $400 to $1,200 at internal labor cost. Higher volumes scale the API line roughly linearly while the labor lines grow in steps.
What drives LLM operating costs besides tokens?
Retrieval infrastructure such as a vector index that someone keeps fresh, the evaluation test set that must be re-run after every prompt change and model upgrade, and the review queue where staff check low-confidence output. In our delivery work the labor lines exceed the token line at every tier below very high volume.
Do model upgrades really force extra work?
Yes. Providers change model behavior without asking, and a prompt tuned on one version can score worse on the next. Every upgrade means re-running your evaluation set, reading the diffs, and re-tuning where scores dropped. Skipping this turns silent drift into customer-facing errors, so budget roughly a day of work per upgrade per workflow.
How can we cut LLM running costs without hurting quality?
Cache repeated context instead of resending it, route simple steps to small cheap models and reserve the flagship model for hard steps, cap how much retrieved text enters each prompt, and batch offline work into off-peak or batch APIs. In our builds these four levers roughly halve token spend on suitable workloads, and caching alone often does most of it.
