Images sent to a model in a loop at full resolution

Image or vision calls inside a loop with no limit on how many.

Rule vision-in-loop-uncappedCost audit

Why it costs money

Images are billed as input tokens, and a full-resolution screenshot can cost around a thousand tokens or more per call. A browser or computer-use agent that sends a fresh screenshot at every step pays that on every iteration.

It compounds when earlier screenshots stay in the history. On step k the request carries k images, so a 20-step run sends 210 images in total instead of 20. Lowering the detail setting or resizing first cuts the per-image cost; dropping old images cuts the count.

Example

Flagged

import base64
from openai import OpenAI

client = OpenAI()


def run(task: str, browser) -> None:
    history = [{"role": "user", "content": task}]
    for _ in range(30):
        shot = base64.b64encode(browser.screenshot()).decode()
        history.append({"role": "user", "content": [
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
        ]})
        response = client.chat.completions.create(model="gpt-4o", max_tokens=300, messages=history)
        action = response.choices[0].message.content
        if action == "DONE":
            return
        history.append({"role": "assistant", "content": action})
        browser.perform(action)

Fixed

import base64
from io import BytesIO
from openai import OpenAI
from PIL import Image

client = OpenAI()


def small_screenshot(browser) -> str:
    img = Image.open(BytesIO(browser.screenshot()))
    img.thumbnail((1024, 1024))
    buf = BytesIO()
    img.save(buf, format="PNG")
    return base64.b64encode(buf.getvalue()).decode()


def run(task: str, browser) -> None:
    actions = []
    for _ in range(30):
        image = {"url": f"data:image/png;base64,{small_screenshot(browser)}", "detail": "low"}
        prompt = task + " Actions so far: " + "; ".join(actions)
        response = client.chat.completions.create(model="gpt-4o", max_tokens=300, messages=[
            {"role": "user", "content": [{"type": "text", "text": prompt}, {"type": "image_url", "image_url": image}]},
        ])
        action = response.choices[0].message.content
        if action == "DONE":
            return
        actions.append(action)
        browser.perform(action)

How to fix it

Downscale images before sending them, and use the provider's low-detail setting where the task allows it. Many agent steps only need to find a button, not read small text.

Send only the current screenshot. Replace earlier images in the history with a short text note of what was done, so the per-step cost stays flat instead of growing with the run.

What the audit detects

Folders where an LLM text call runs inside a loop, and files in those folders that build image content: image_url parts, "type": "image" blocks, base64 data URLs, screenshots or inline_data.
Only files with no size control at all: no detail set to low, and no resize, thumbnail, downscale, image-size setting, sharp() or PIL usage.
Files whose own calls generate images, re-export files, and test, example and docs folders are skipped. One finding per folder. It does not prove the image and the loop are on the same code path.