How to reduce LLM API costs: patterns that inflate the bill
The code patterns that inflate an LLM API bill: missing output caps, oversized models, resent history, unbounded loops, no budget guard, and how to fix each.
To reduce LLM API costs, fix the code patterns that multiply spend before you negotiate on price per token. The patterns that matter most are calls with no output cap, frontier models doing simple jobs, conversations re-sent in full on every turn, loops and retries with no limit, bulk work sent one request at a time, the same answer paid for twice, and paid endpoints that anyone can call. Each of them can make a bill several times larger than it needs to be, and none of them shows up as an error.
This guide explains how each pattern inflates cost, how to recognise it in code, and what the fix looks like. Several patterns have their own longer guide, linked from the relevant section.
Where the money goes
Every provider bills roughly the same way: for each call, input tokens times the input price plus output tokens times the output price. Output tokens are usually priced several times higher than input tokens. Monthly spend is that per-call cost times the number of calls. So there are only three levers: the number of calls, the tokens per call, and the price per token. Every pattern below inflates one of them.
Illustrative arithmetic (prices are made up for the example):
input price = $3.00 per 1M tokens
output price = $15.00 per 1M tokens
one call = 4,000 input tokens + 800 output tokens
= 0.004 * $3 + 0.0008 * $15 = $0.012 + $0.012 = $0.024
100,000 calls a month = $2,400
Halve the calls, the tokens, or the price and the bill halves.
Double any of them by accident and it doubles.1. Calls with no output cap
A call without max_tokens (or max_completion_tokens, max_output_tokens, maxOutputTokens, depending on the SDK) lets the model decide how long to talk. Most of the time it stops early. Occasionally it produces a very long answer, repeats itself, or gets stuck in a list, and you pay output prices for all of it. Set a cap that fits the task, and put a sensible default in the shared client so new call sites inherit it. See no-max-tokens.
import OpenAI from "openai";
const openai = new OpenAI();
// A one-line classification never needs more than a few tokens back.
const res = await openai.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: prompt }],
max_completion_tokens: 20,
});2. A frontier model on a small job
Classification, extraction, routing, short rewrites and yes/no checks rarely need the largest model in a family. The price gap between a flagship model and its small sibling is often an order of magnitude. The honest way to decide is to run both on a sample of your own traffic and compare. Choosing a cheaper LLM model covers right-sizing, routers and cascades, and what the research numbers do and do not promise. The rule is model-oversized.
3. Re-sending the whole conversation
Chat APIs are stateless, so most chat code appends every message to a list and sends the whole list each turn. Input tokens per turn grow linearly with the length of the conversation, which makes the total for the conversation grow with its square. The fix is a sliding window, a running summary, or both. See full-history-resend.
Example: 50 turns, about 300 tokens added per turn.
Full history: 300 * (1 + 2 + ... + 50) = 300 * 1,275 = 382,500 input tokens
Last 10 turns: 300 * (1 + ... + 10) + 300 * 10 * 40 = 16,500 + 120,000 = 136,500
Roughly 64% fewer input tokens for the same conversation.MAX_TURNS = 10
def build_messages(system_prompt, summary, history):
recent = history[-MAX_TURNS * 2:] # user + assistant per turn
messages = [{"role": "system", "content": system_prompt}]
if summary:
messages.append({"role": "system", "content": "Earlier in this conversation: " + summary})
return messages + recent4. Loops, retries and jobs with no limit
The largest surprise bills usually come from something running more times than anyone expected: an agent loop that never reaches a final answer, a retry wrapper with no attempt cap, a polling loop that keeps a paid job alive after the caller has gone, or a scheduled job that calls a model every few minutes whether or not anyone is using the app. The fixes are bounds: a maximum iteration count, a hard retry cap with exponential backoff and jitter, a deadline plus a cancel call, and a per-run budget for scheduled work. How to stop runaway AI agent loops goes through each one. The rules are agent-loop-unbounded, retry-storm, orphaned-async-job and recurring-job-llm.
5. No spending cap on the path to a paid call
Bounds on individual loops help, but something has to stop spend when every other assumption fails. That means a per-run or per-user budget in the code, rate limits and authentication on endpoints that trigger model calls, and validation of request fields before they reach the model. A common near-miss is a guard that only exists when an environment variable is set, so it is off in exactly the deployments nobody is watching. LLM spending limits covers the layers in detail. Relevant rules: no-budget-guard, guard-conditional-on-env, unauthenticated-llm-endpoint and unvalidated-input-to-paid-call.
6. Paying twice for the same answer
There are two different caches, and they are often confused. Provider prompt caching discounts a repeated prompt prefix, but it can cost more than no caching if the cached part changes on every call, and some providers bill storage by the hour for caches you forget to delete. Response caching stores the result of a deterministic call so a duplicate request never reaches the provider at all. Prompt caching vs response caching explains when each one pays off. Rules: cache-write-churn, cache-storage-leak and result-cache-candidate.
7. One request per item when a batch API exists
Backfills, nightly enrichment, evaluation runs and bulk embedding jobs often loop over items and make one synchronous call each. If the results are not needed within seconds, OpenAI and Anthropic both offer batch endpoints that, at the time of writing, charge half the normal price in exchange for completing within a window of up to 24 hours. Embedding endpoints also accept many inputs per request. See sync-batch-candidate.
8. Hidden multipliers
Some costs multiply quietly because they sit next to the main call rather than in it:
- Auxiliary calls: moderation, title generation, query rewriting and summarisation steps that each add a model call to every request. Five calls per user message is a 5x multiplier nobody designed. See auxiliary-call-multiplication.
- Reasoning budgets: a high
budget_tokensorreasoning_effortset once on a helper that then runs inside a loop. Reasoning tokens are billed as output. See reasoning-budget-in-loop. - Images in loops: full-resolution screenshots sent on every step of a browser agent, often with every previous screenshot still in the history. See vision-in-loop-uncapped.
- Sticky fallbacks: an error handler that swaps to a pricier model and never switches back, so one timeout changes the price of the rest of the run. See sticky-fallback-model.
9. Keys that let someone else spend your money
A provider key in a browser bundle (a NEXT_PUBLIC_ or VITE_ variable, or dangerouslyAllowBrowser: true) or committed to a repository is spend you do not control. Leaked keys are scraped and abused quickly, and deleting the line does not help once it is in git history: rotate the key. See client-side-key-exposure and hardcoded-api-key.
Find the patterns before the bill does
Provider dashboards tell you that spend went up. They do not tell you which line of code did it. Runtime tracing tools close part of that gap once traffic is flowing. Reading the code closes the rest before deploy, including paths that rarely run, like a nightly job or an error handler. LLM cost monitoring vs static analysis compares the two approaches and argues for using both.
How IntentiAI helps
IntentiAI is an open-source (MIT) command-line tool that finds every LLM call site in a repository and checks it against 21 cost rules covering the patterns above. Run npx intentiai from the repository root. It needs no API key and makes no model calls. It uploads findings only (rule, file, line, summary, fix), never source, and prints a private report link; --local uploads nothing. TypeScript, JavaScript and Python get all 21 rules; Go, Java, Ruby and PHP get every call site and 8 of the rules.
It is a static audit, so it cannot see real traffic, and every dollar figure is an estimate with its assumptions stated. On the project's own measurements it found 98.6% of model calls across seven hand-labelled open-source apps, and 45% of hand-labelled cost problems in five apps the rules were not written against. Read the rule list or the docs.