IntentiAI

LLM spending limits: budget guards and rate limits in code

How to put spending caps, per-user budgets and rate limits in front of paid LLM calls, why env-conditional guards fail open, and how to make guards fail closed.

4 min read

An LLM spending limit that works has to live in your code, in front of every paid call, and has to be on by default. Provider-level limits are a useful last line, but they are coarse: they apply to a whole organisation or project, may only send an alert, and stop every user at once when they trip. The layers that protect a production app are a per-run budget, a per-user or per-tenant quota, request rate limits, authentication on any endpoint that triggers a model call, and input validation before the call.

OWASP names the failure this prevents in its 2025 Top 10 for LLM applications as LLM10, Unbounded Consumption, which includes denial of wallet: an attacker, or a bug, drives paid inference until the bill becomes the outage.

Rate limits and spend limits are different controls

A rate limit counts requests. A spend limit counts money. They fail differently, and you need both. One request with a 100,000-token document and a high output cap can cost more than a thousand short requests, so a limit of 60 requests per minute says little about dollars. Conversely, a spend limit alone does not stop a script from hammering an endpoint with cheap requests that exhaust your provider rate limit for everyone else.

The layers

  1. Authentication. An endpoint anyone can call is an endpoint anyone can spend through. Require a user identity on every route that reaches a model. See unauthenticated-llm-endpoint.
  2. Input validation. Check request fields against an allowlist or schema before the call: which model, how many tokens, how many items. A client-supplied model field passed straight through lets a caller pick your most expensive model. See unvalidated-input-to-paid-call.
  3. Rate limit per identity. Requests per minute per user or API key, enforced in shared storage so it holds across instances.
  4. Spend quota per identity. Dollars or tokens per day or month per user or tenant.
  5. Budget per run. A ceiling on a single agent run or background job, so one bad run cannot use a whole user's quota. See How to stop runaway AI agent loops.
  6. Output caps. max_tokens on every call, so the worst case for one call is known. See no-max-tokens.
  7. Provider limits and alerts. Organisation or project spend limits and usage alerts in each provider console, as the backstop when everything above fails.

A per-user spend quota

The tricky part is that you only know the exact cost after the call returns. A workable pattern is to reserve the worst case before the call (input tokens plus the output cap, times the price) and settle to the actual usage afterwards. The reservation stops many concurrent requests from each seeing a nearly empty budget and all going through.

import { Redis } from "ioredis";

const redis = new Redis(process.env.REDIS_URL!);
const DAILY_LIMIT_MICRO_USD = 2_000_000; // $2.00 per user per day

function key(userId: string) {
  return "spend:" + userId + ":" + new Date().toISOString().slice(0, 10);
}

export async function reserve(userId: string, worstCaseMicroUsd: number) {
  const k = key(userId);
  const total = await redis.incrby(k, worstCaseMicroUsd);
  await redis.expire(k, 60 * 60 * 48);
  if (total > DAILY_LIMIT_MICRO_USD) {
    await redis.decrby(k, worstCaseMicroUsd);
    throw new QuotaExceeded(userId);
  }
}

export async function settle(userId: string, reservedMicroUsd: number, actualMicroUsd: number) {
  await redis.decrby(key(userId), reservedMicroUsd - actualMicroUsd);
}

Wrap your model client once so every call goes through reserve and settle. Guards added call site by call site get missed on the next feature. Store money as integers (micro-dollars here) to avoid floating-point drift, and settle in a finally block so a failed call releases its reservation.

The guard that is only there sometimes

A common near-miss is a guard that exists in the code but is conditional on configuration. It is on in the maintainer's production and off everywhere else: preview deployments, forks, self-hosted installs, a new region where someone forgot one variable.

// Fails open: with no REDIS_URL there is no limit at all.
const limiter = process.env.REDIS_URL ? new RateLimiter(process.env.REDIS_URL) : null;

export async function POST(req: Request) {
  const user = await getUser(req);
  if (limiter) await limiter.check(user.id);
  return streamCompletion(await req.json());
}

The same shape appears as if (process.env.ENABLE_QUOTAS === "true"), a max_budget setting that defaults to None, or a quota read from a database row that does not exist until an admin creates it. The OWASP framing is the right one: consumption is governed by the deployed configuration, not by the presence of guard code. Two fixes, and you usually want both:

  • Fail closed with a local default. If the shared store is missing, fall back to an in-memory limiter with conservative limits rather than none. It is per-instance and imperfect, but it bounds the worst case.
  • Refuse to boot in production without the guard. Check required configuration at startup and exit with a clear message. A crash at deploy time is cheaper than a quiet month without limits.
import os

def build_limiter():
    url = os.environ.get("REDIS_URL")
    if url:
        return RedisLimiter(url, per_minute=20, daily_usd=2.0)
    if os.environ.get("APP_ENV") == "production":
        raise RuntimeError("REDIS_URL is required in production: spend limits cannot be enforced without it")
    return InMemoryLimiter(per_minute=20, daily_usd=2.0)

limiter = build_limiter()

Keys outside your control

All of the above assumes calls go through your server. A provider key shipped to the browser, through a public build-time variable or a dangerouslyAllowBrowser style flag, bypasses every layer: anyone can copy it and call the provider directly. The same goes for a key committed to a repository. Keep keys server-side, and rotate any key that has been exposed, since removing it from the latest commit does not remove it from history. See client-side-key-exposure and hardcoded-api-key.

Choosing the numbers

Set quotas from observed usage rather than guesses. Look at the cost per active user per day over a few weeks, set the default quota well above the heavy end of normal, and make it configurable per plan. Alert on users who hit the quota repeatedly: they are either your best customers or someone scripting your endpoint, and you want to know which. Revisit the numbers whenever you change models, because a model change can move cost per request by an order of magnitude in either direction.

How IntentiAI helps

The cost audit checks for these gaps statically. no-budget-guard reports when no dollar or token cap exists anywhere on the path to a paid call, and distinguishes a guard that is absent from one that exists but ships disabled. guard-conditional-on-env finds guards that only exist when certain environment variables are set. The endpoint, input and key rules above cover the other layers. Run npx intentiai from the repository root; --local keeps the results on your machine.

A static check cannot see limits enforced outside the repository, such as an API gateway or a proxy, and each finding says so. On the change side, Ship Check's removed-guard check flags a diff that deletes a validation, permission or rate-limit call without a replacement, which is a mistake coding agents make when a guard gets in the way of a test. See /cost-audit, /ship-check and the docs.

More guides

LLM spending limits: budget guards and rate limits in code | IntentiAI