Cost per Successful Task LLM: Why Cheap‑Per‑Token Models Can Be More Expensive

Introduction – Defining “cost per successful task”

When teams evaluate large language models (LLMs) they often start with price per token as the primary metric. That number is seductive: a 0.5 ¢/1K token model looks cheaper than a 5 ¢/1K token model. But the real cost of a task is rarely captured by token price alone.

Cost per successful task is the amount of spend required to get a completed, correct outcome from an LLM‑driven workflow. It includes:

Model call spend (tokens used)
Retry overhead (failed attempts, additional tokens)
Human review or correction time
Infrastructure latency that may force extra calls

Consequently, a model with a low per‑token price can end up more expensive per successful task when it frequently fails, needs multiple attempts, or forces downstream human intervention. Understanding this metric is essential for any commercial AI deployment that wants predictable budgets and reliable performance.

The hidden costs of cheap models

Cheaper models often have higher error rates on complex reasoning, multi‑step planning, or nuanced context handling. When they fail, the workflow must either:

Retry the same model – consuming additional tokens and compute.
Escalate to a more capable (and pricier) model – after several unsuccessful passes.
Engage a human reviewer – adding labor cost and slowing throughput.

Real‑world example: reasoning tasks

Consider a financial analyst agent that must extract and reconcile data from a quarterly earnings release. Using GPT‑4o‑mini (≈0.5 ¢/1K tokens) the agent may need 3–4 attempts to correctly parse the table and generate a summary. Each attempt costs roughly 2 K tokens, so a successful run can consume 6–8 K tokens → roughly 3 ¢ in model spend alone, plus the cost of a human check.

Switching to GPT‑4o (≈5 ¢/1K tokens) the same task often succeeds in a single pass, using about 1.5 K tokens → ≈7.5 ¢. Although the per‑token price is ten times higher, the total cost per successful task is comparable, and the latency is far lower.

The Stanford study on LLM agent costs found that agents can incur up to 10× more cost in retries and error handling compared to the base token cost, underscoring the importance of measuring end‑to‑end task cost rather than raw token price.

Model	Price per 1K tokens	Typical tokens per attempt	Expected attempts for complex task	Approx. cost per successful task
GPT‑4o‑mini	0.5 ¢	2 K	3–4	~3 ¢ (plus possible human review)
GPT‑4o	5 ¢	1.5 K	1	~7.5 ¢
Claude‑3‑Haiku	2 ¢	1.8 K	1–2	~4 ¢ (varies)

These numbers illustrate that cheap LLM vs expensive LLM total cost is not a simple comparison of per‑token rates; the successful task outcome drives the true expense.

How to measure cost per successful task

A robust measurement framework combines observability with a simple formula:

Cost per successful task = (Total spend on model calls
                           + Retry token cost
                           + Human review cost)
                           / Number of successful task completions

Key observability signals to capture:

Token usage per call (input + output)
Latency (time to first token, time to complete)
Failure rate (when the model returns “I don’t know” or an invalid format)
Human intervention flag (whether a reviewer had to correct the output)

Without these metrics, teams cannot distinguish between a model that is cheap per token and one that is cheap per successful task. This is where LLM observability cost metrics become indispensable.

Harpd’s approach to observability

The open‑source @harpd/observe package (MIT