How to Reduce OpenAI API Cost

If you've shipped an AI agent or chatbot and watched the API bill come in at 5-10x your estimate, you're not doing anything unusually wrong. It's one of the most common surprises in production LLM work, and it almost always comes down to the same handful of causes , none of which show up until you're already live.

This isn't a "switch to a cheaper model" post. That trade usually just swaps cost for reliability problems. The actual fix is architectural: stop sending every request the same bloated context, and let the request itself determine what it needs.

Why the bill is higher than the math suggested

Three things quietly multiply cost beyond a simple tokens-times-price estimate.

1. Tool calls are billed requests, not free side effects

If your agent uses function calling, a single user message rarely maps to a single API call. "Move my meeting to tomorrow" might trigger:

1. An initial call to interpret the request
2. A tool call to look up the calendar event
3. A tool call to update it
4. A confirmation step
5. A final response back to the user

That's four or five billed calls behind what looks, from the outside, like one interaction. If you budgeted per user message instead of per underlying API call, this gap alone explains most of a 5-10x overshoot.

2. Every request defaults to the flagship model

It's tempting to route everything through the most capable model available so you don't have to debug flaky outputs. But that treats "what's on my calendar today" and "analyze this 40-page contract" as if they need the same reasoning power. They don't, and paying flagship pricing for the former is pure waste.

3. The system prompt and tool list are sent in full, every time

Every call carries your entire system prompt and entire tool schema as input tokens , even when the request only touches a fraction of it. A 15-tool, multi-thousand-token system prompt on every single call adds up fast, and ironically it also makes cheaper models less reliable, since they now have to sift through instructions and tools that don't apply. That pushes teams toward the expensive model, which is the trap in the first place.

The pattern: classify first, assemble second, route third

The fix that shows up repeatedly in production agents is a lightweight routing layer that runs before the "real" call, deciding exactly what context and which model a given request actually needs.

Step 1 : Classify intent with a cheap, fast model

Run the incoming request through an inexpensive model first, with one job: figure out what kind of request this is, roughly how complex it looks, and which tool categories it'll likely need.

async function classifyIntent(message: string) {
  const result = await callModel({
    model: "gemini-2.0-flash", // fast, cheap, this call runs on every request
    system: INTENT_CLASSIFIER_PROMPT,
    input: message,
  });

  return result as {
    intentType: "lookup" | "scheduling" | "analysis" | "deletion";
    complexity: "simple" | "moderate" | "complex";
    toolCategories: string[];
  };
}

This call costs a fraction of a cent and removes the guesswork from everything downstream.

Step 2 : Assemble the system prompt from modules, not one giant block

Instead of one static prompt containing every rule and edge case, break instructions into small modules ( scheduling, deletion, timezone handling, confirmation rules ) and assemble only what the classified intent needs.

const PROMPT_MODULES = {
  core: CORE_INSTRUCTIONS,       // always included
  scheduling: SCHEDULING_RULES,
  deletion: DELETION_SAFEGUARDS,
  timezone: TIMEZONE_HANDLING,
} as const;

function buildSystemPrompt(intent: ClassifiedIntent) {
  const modules = [PROMPT_MODULES.core];

  if (intent.intentType === "scheduling") modules.push(PROMPT_MODULES.scheduling);
  if (intent.intentType === "deletion") modules.push(PROMPT_MODULES.deletion);
  if (intent.requiresTimezone) modules.push(PROMPT_MODULES.timezone);

  return modules.join("\n\n");
}

Teams that adopt this typically see system prompt size drop from something like 20,000+ tokens to 2,000-5,000 tokens per call ( applied to every request, not a one-time saving).

Step 3 : Send only the relevant tool definitions

Same logic applies to function/tool schemas. If your agent has 15-20 tools defined, a given request usually needs 2-4 of them. Group tools by category and attach only what the classified intent calls for.

const TOOL_GROUPS = {
  search: [lookupEventTool, searchContactsTool],
  scheduling: [createEventTool, updateEventTool, cancelEventTool],
  dataModification: [deleteRecordTool, archiveRecordTool],
};

function selectTools(intent: ClassifiedIntent) {
  return intent.toolCategories.flatMap((category) => TOOL_GROUPS[category] ?? []);
}

This alone commonly cuts tool-schema overhead by 50-70%.

Step 4 : Route to a model that matches the actual complexity

The insight that's easy to miss: smaller models don't fail because they're incapable , they fail because they're handed too much irrelevant context and too many tools to reliably pick from. Give a small model exactly what it needs, and its reliability on that narrower task improves significantly, often closing most of the gap with larger models.

Why three cheap calls can beat one expensive call

It sounds counterintuitive that classify → assemble → execute (two or three model calls) would cost less than one call to a single expensive model. In practice, the combined cost of a few small, targeted calls is usually a fraction of one large call carrying a bloated prompt and full tool list. You're paying for precision instead of paying for redundancy , and because each smaller call gets a narrower, well-scoped job, latency often improves too, not just cost.

Other levers worth stacking on top

• Cache aggressively. If your system prompt prefix repeats across calls, prompt caching avoids paying full price for input tokens you've already sent.
• Set hard output token limits. Uncapped output length is a quiet, constant cost leak.
• Use retrieval instead of stuffing context. If you're pasting entire documents or full conversation history into every prompt "just in case," a retrieval step that pulls only relevant snippets usually cuts input tokens dramatically with no quality loss.
• Batch non-urgent work. Summarization jobs, nightly reports, and bulk classification don't need real-time responses , batch APIs process these asynchronously at a meaningful discount.

Don't skip evaluation

Every optimization here reduces the context a model sees, which naturally raises the question: does accuracy hold up? The only reliable way to know is an automated evaluation suite , a set of representative test scenarios you re-run every time a prompt, model, or routing rule changes.

The takeaway

Cutting LLM API costs isn't primarily about swapping to a cheaper model , that trades cost for reliability problems more often than it saves money outright. The durable fix is architectural: classify before you execute, assemble only the context and tools a given request actually needs, and reserve expensive models for requests that genuinely require them. Done well, this pattern routinely holds ( or even improves ) output quality, because every model in the pipeline is finally being asked to do a job it's actually suited for.

If you're mid-build and want a second pair of eyes on where your own architecture is leaking cost, get in touch , this is exactly the kind of thing worth catching before it shows up on an invoice.