LLM call with no output token limit

A call with no output cap on a model where long answers are expensive.

Rule no-max-tokensCost audit

Why it costs money

Output tokens are priced at several times the input rate on most current models, and a response with no cap can run to the model's full output limit. Most of the time the model stops early on its own. The cost problem is the tail: a prompt that triggers a rambling answer, a repeated phrase, or a list that never ends.

As an example, if a summary normally runs 200 tokens and one request in fifty runs to 8,000, those outliers alone roughly double the output bill for that endpoint. A cap turns the worst case into a known number.

Example

from openai import OpenAI

client = OpenAI()


def summarize_ticket(ticket_text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Summarize this support ticket for an agent."},
            {"role": "user", "content": ticket_text},
        ],
    )
    return response.choices[0].message.content

from openai import OpenAI

client = OpenAI()


def summarize_ticket(ticket_text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        max_tokens=300,
        messages=[
            {"role": "system", "content": "Summarize this support ticket for an agent in at most five sentences."},
            {"role": "user", "content": ticket_text},
        ],
    )
    return response.choices[0].message.content

How to fix it

Set an output cap that fits the task, with some headroom over the longest good answer you have seen. Also ask for the length you want in the prompt, so the model finishes naturally instead of being cut off.

If you route many providers through one client, set a default cap per model in that shared config rather than at every call site. Check finish_reason (or stop_reason) so truncated answers are handled rather than shown as complete.

What the audit detects

• Text completion calls that set none of max_tokens, max_completion_tokens, max_output_tokens, maxOutputTokens or max_new_tokens. Caps inside config objects (inferenceConfig, GenerateContentConfig) and caps passed through a spread or kwargs defined in the same file count as set.
• Embedding, image and audio calls are excluded.
• Findings are grouped per file with the line numbers. When more than eight files are affected, it reports one repo-level finding instead.
• It is a warning when other calls in the same repo do set a cap (the uncapped ones are outliers), and informational otherwise.