Research you can verify, not just read
Harpd publishes open datasets, reproducible benchmark methods and documented methodology. The point is that anyone — a competitor, an auditor, or you — can reproduce our numbers.
We publish the raw data and the exact methods behind every AI cost and model-quality claim. Where a number is modeled or not yet measured, we say so — we never present estimates as results.
- Canonical prices: /data/llm-pricing.json
- Spreadsheet form: /data/llm-pricing.csv
- Methods: four documented methodologies
- Open source: github.com/harpd-dev
What we publish
Everything below is real, machine-readable, and referenced from the pages that make claims.
Open pricing dataset
One JSON/CSV record per model, sourced from official provider pricing pages, each with a sourceUrl and verifiedAt timestamp. The canonical source of truth for every price on Harpd.
Documented methodology
Four methodologies — model pricing, model replacement, cost per successful task, and reproducible benchmarks — each written down so the measurement is repeatable and auditable.
Read methods →Reproducible benchmarks
Real-task evaluations with published success definitions, the prices used, and CSV/JSON artifacts. Runner repos are open so the whole run can be replayed.
Benchmark method →What we don't (yet)
Honesty is the entire point. We are explicit about the gaps.
- We do not publish measured benchmark results yet — the benchmark is in a PLANNED / MODELED stage (target ship 2026-09-15). Nothing on /benchmarks/ is a result.
- We do not fabricate model prices. Unknown prices are marked as unavailable, never guessed.
- We do not present modeled estimates as measured results. Modeled numbers on the site are visually distinct and labeled "modeled".
- We do not claim a model is "safe to switch" without a real-task quality gate passing on your workload.
Open-source tooling
The methods above are backed by public repositories under github.com/harpd-dev.
observe
Runtime instrumentation for measuring AI workload cost and token flow in production.
agent-budget-policy
Declarative per-agent budget and guardrail policies for controlling AI spend.
agent-transaction-audit-schema
A shared schema for auditing agent-initiated transactions and reconciling spend.
mcp-paid-tool-starter
A starter for paid MCP tools with metering and pricing wired in from day one.