From $10K to $1K/Month: Optimizing AI Agent Spending for a SaaS Startup

A 12-person SaaS startup was burning $10,000 a month on AI agent infrastructure. Twelve months later, that number was $1,000 — and the product was better. No headcount cuts. No feature removals. No compromise on quality. Just a systematic approach to AI agent spending that every SaaS team can replicate.

This is the story of how they did it, step by step.

The Starting Point: $10K/Month and Rising

The startup built an AI-powered document analysis tool for legal teams. Customers uploaded contracts, and the product used a chain of LLM calls to extract key terms, flag risky clauses, and generate summaries. The agent architecture was sound — the cost structure was not.

Here’s what the $10K/month breakdown looked like:

Category	Monthly Cost	% of Total
Primary LLM (GPT-4 class)	$6,200	62%
Embedding model	$1,800	18%
Web search API	$900	9%
Document parsing	$700	7%
Monitoring & logging	$400	4%
Total	$10,000	100%

The founders knew the numbers were unsustainable. At their pricing tier ($49/month per seat), they needed fewer than 200 paying customers to cover AI costs alone — before hosting, salaries, or marketing. The math didn’t work.

But they also knew the AI capabilities were the product. Cutting costs meant cutting quality, right?

Wrong. It meant cutting waste.

Step 1: Map Where the Money Goes

The first insight came from a simple audit. They traced every dollar to a specific agent action and asked: Did this call need to happen?

The answer was surprising. 43% of their LLM calls were retries. When the primary model returned a parsing error or an incomplete extraction, the agent automatically retried — up to five times — with the same expensive model. Each retry was a full-priced call.

Another 18% were “exploration” calls. The agent would try multiple approaches to extract the same data point, picking the result with the highest confidence score. Three calls to get one answer.

The audit revealed the core problem: they were optimizing for accuracy but paying for redundancy. The agent was over-engineering every task because no one had defined when “good enough” was good enough.

The Audit Framework

They built a simple spreadsheet to map cost drivers:

For each agent action:
1. What model was called?
2. How many times per customer interaction?
3. What was the success rate on the first attempt?
4. What was the average retry count?
5. What was the cost per successful extraction?

This framework took one engineer two days to build and revealed $4,300/month in unnecessary spending — before any code changes.

Step 2: Implement Budget Policies

The second insight was structural. Every agent had the same budget: unlimited. There was no per-task, per-agent, or per-user cap. A single customer running a batch analysis of 500 contracts could generate $200 in API costs in one session.

They deployed @harpd/agent-budget-policy to define tiered budgets:

import { defineBudget } from '@harpd/agent-budget-policy';

const policy = defineBudget({
  perCall: { usdMax: 0.05 },
  perAgent: { '*': { usd: 15, window: '1d' } },
  perEndpoint: { '/api/extract': { usd: 50, window: '1d' } },
  perTool: { 'web-search': { usd: 2, window: '1h' } },
});

The impact was immediate. When an agent hit its per-call cap, it received a structured denial and switched to a cheaper model instead of retrying the expensive one. When a user’s daily spend approached $15, the system paused and notified the team.

Cost reduction from budget policies: $2,100/month (from $10K to $7,900)

Step 3: Route to the Right Model

The third insight was about model selection. They were using a GPT-4 class model for every task — simple extraction, complex reasoning, classification, summarization. But not every task needs the most expensive model.

They implemented a three-tier model routing strategy:

Task Complexity	Model Tier	Cost per 1K Tokens	Used For
Simple extraction	Haiku-class	$0.00025	Pulling dates, names, dollar amounts
Moderate reasoning	Sonnet-class	$0.003	Clause risk analysis, summary generation
Complex analysis	GPT-4 class	$0.03	Multi-document comparison, edge cases

The routing logic was straightforward. Each task was classified by complexity before invocation, and the appropriate model was selected. 78% of tasks fell into the “simple” category — tasks that were being routed to the most expensive model.

They used a simple classifier to make routing decisions:

function routeModel(task: AgentTask): ModelTier {
  if (task.type === 'extraction' && task.entities.length <= 3) {
    return 'simple';
  }
  if (task.type === 'analysis' && task.documents.length > 2) {
    return 'complex';
  }
  return 'moderate';
}

Cost reduction from model routing: $3,200/month (from $7,900 to $4,700)

Step 4: Eliminate Retry Waste

The fourth insight was about error handling. Their retry logic was aggressive — five retries with exponential backoff. But most retries weren’t fixing errors; they were trying to improve results that were already acceptable.

They implemented a two-part fix:

1. First-attempt quality scoring. Before retrying, the agent evaluated whether the first result met a minimum quality threshold. If it scored above 0.85, the result was accepted — even if it wasn’t perfect.

2. Escalation over retry. After two failed attempts, the agent stopped retrying and escalated to a human reviewer. This prevented infinite loops and provided training data for future improvements.

const result = await extractWithModel(task, primaryModel);
const score = evaluateQuality(result, task.expectedSchema);

if (score >= 0.85) {
  return result; // Accept — don't optimize further
}

if (retryCount < 2) {
  return retryWithFallbackModel(task, retryCount + 1);
}

return escalateToHuman(task, result, score); // Stop burning money

Cost reduction from retry optimization: $1,800/month (from $4,700 to $2,900)

Step 5: Cache Aggressively

The fifth insight was about caching. Many of their customers analyzed similar document types — NDAs, SaaS agreements, employment contracts. The same clauses appeared across thousands of documents, yet every analysis was a fresh API call.

They implemented a two-layer caching strategy:

Semantic caching: Embeddings of extracted clauses were stored in a vector database. Before making an API call, the agent checked whether a semantically similar extraction had already been performed. If the similarity score exceeded 0.95, the cached result was returned.

Template caching: Common document structures were recognized and matched to templates. An NDA extraction used the same template every time, avoiding a full LLM analysis.

async function cachedExtract(task: AgentTask): Promise<Extraction> {
  const cacheKey = generateSemanticKey(task.document);
  const cached = await vectorDB.search(cacheKey, { threshold: 0.95 });
  
  if (cached) {
    metrics.increment('cache.hit');
    return cached.result;
  }
  
  const result = await extractWithRouting(task);
  await vectorDB.insert(cacheKey, result);
  metrics.increment('cache.miss');
  return result;
}

Cost reduction from caching: $1,100/month (from $2,900 to $1,800)

Step 6: Monitor and Iterate

The sixth insight was cultural. Cost optimization isn’t a one-time project — it’s an ongoing practice. They built a real-time cost dashboard using @harpd/x402-logging-middleware to track spending per agent, per task, and per customer.

The dashboard surfaced three key metrics:

• Cost per successful extraction — the true cost of doing business
• Retry rate — how often the agent was burning money on failed attempts
• Cache hit rate — how effectively they were reusing prior work

Every week, the team reviewed the dashboard and asked: Where did we waste money this week? The answers drove incremental improvements that compounded over time.

Cost reduction from monitoring: $800/month (from $1,800 to $1,000)

The Final Numbers

Category	Before	After	Savings
Primary LLM	$6,200	$480	$5,720 (92%)
Embedding model	$1,800	$220	$1,580 (88%)
Web search API	$900	$180	$720 (80%)
Document parsing	$700	$120	$580 (83%)
Monitoring & logging	$400	$0	$400 (100%)
Total	$10,000	$1,000	$9,000 (90%)

And here’s the part that matters: customer satisfaction scores went up. Faster response times from caching. More consistent results from model routing. Better error handling from escalation logic. The product improved as costs dropped.

What Made This Work

Three principles drove the success:

1. Measure before optimizing. The two-day audit revealed more savings than weeks of code changes. You can’t fix what you can’t see.

2. Enforce at the transaction level. Budget policies that evaluate before each API call catch waste in real time. Post-hoc analytics tell you what happened; pre-transaction checks prevent it.

3. Make cost a first-class metric. The weekly dashboard reviews made cost optimization a team habit, not a one-time project. Every engineer started asking “does this need to be expensive?” before reaching for the premium model.

How to Start Today

You don’t need to rebuild your stack to replicate these results. Here’s the minimum viable path:

1. Audit your spending. Map every API call to a cost. Find your retries and redundancies.
2. Add budget policies. Use @harpd/agent-budget-policy to set per-call and per-agent limits.
3. Implement model routing. Not every task needs the most expensive model.
4. Cache what you can. Semantic caching eliminates redundant calls.
5. Monitor in real time. Build a cost dashboard and review it weekly.

The tools are open source. The principles are proven. The only variable is whether you start now or wait for the next invoice shock.

---

Ready to optimize your AI agent spending? Start with Harpd’s open-source tools — agent budget policies, model routing, and real-time cost monitoring — or read the payment guardrails guide to build a complete cost control system.