No spend cap on LLM calls

No dollar or token cap anywhere on the path to a paid call.

Rule no-budget-guardCost audit

Why it costs money

A budget guard is the control that still works after something else has gone wrong. Without one, a loop that does not stop, a retry storm or a traffic spike keeps sending paid requests until someone reads the invoice or a provider limit trips.

Limits in a provider dashboard help, but they are usually monthly, account-wide and applied after the fact. A cap in code can stop one run, one user or one job at the moment it crosses its allowance, before the next call goes out.

A common variation is spend that is measured but never enforced: the code has a pricing table and adds up cost per request, yet nothing compares the total to a limit and stops.

Example

import OpenAI from "openai";

const openai = new OpenAI();

export async function runResearch(question: string) {
  const notes: string[] = [];
  for (const step of planSteps(question)) {
    const res = await openai.chat.completions.create({
      model: "gpt-4o",
      max_tokens: 800,
      messages: [{ role: "user", content: step }],
    });
    notes.push(res.choices[0].message.content ?? "");
  }
  return notes;
}

import OpenAI from "openai";

const openai = new OpenAI();
const TOKEN_BUDGET = 50_000; // per research run

export async function runResearch(question: string) {
  const notes: string[] = [];
  let tokensUsed = 0;
  for (const step of planSteps(question)) {
    if (tokensUsed >= TOKEN_BUDGET) {
      throw new Error("Token budget exceeded for this run");
    }
    const res = await openai.chat.completions.create({
      model: "gpt-4o",
      max_tokens: 800,
      messages: [{ role: "user", content: step }],
    });
    tokensUsed += res.usage?.total_tokens ?? 0;
    notes.push(res.choices[0].message.content ?? "");
  }
  return notes;
}

How to fix it

Add a per-run, per-user or per-day budget in tokens or dollars, count usage from each response, and refuse the next call once the budget is spent. The usage field that every major SDK returns is enough to do this without a separate billing integration.

If a guard already exists but defaults to off (None, -1, unlimited), ship it on with a sensible default and let operators raise it deliberately.

What the audit detects

• Runs only on repos that contain at least one LLM call site.
• Searches every source file for budget-style names such as max_cost, max_budget, token_budget, spend_limit, cost_cap, circuit_breaker, kill_switch and check_budget. One active occurrence is enough to silence the rule.
• If every occurrence is commented out or set to an off value (None, null, -1, 0, false, unlimited, inf), it reports a guard that exists but is off by default and points at that line.
• If pricing or usage-cost code exists but no guard does, it reports spend as metered but not enforced. Limits enforced by a gateway or infrastructure outside the repo are not visible to it.

Research

When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents. Cost exhaustion is one of the main harms of agent loops that fail to terminate. A spend limit is the backstop when loop detection misses a case.