all free tools

rag vs long context cost calculator

For questions over a fixed reference corpus, such as a policy manual, a tariff nomenclature or a contract library, there are two ways to give the model what it needs. Put the whole corpus in every prompt and let prompt caching absorb the repeat, or retrieve the few chunks that look relevant and pay for embeddings and a vector store. Put in your workload and see which costs less, where the two cross, and the arithmetic behind both.

free, no sign-up, runs in your browseropen source on GitHub
the workload
long context
rag

rag is cheaper by $802 a month

$1,604long context per month
$802rag per month
$0.0802long context per query
$0.0401rag per query, fixed costs included
547break-even, queries a month
$4,000rag one-off cost

the arithmetic

  • Long context (Claude Sonnet 5.5, 301,500-token prompt): 1,200 cache writes (40 a day × 30, at most one per query) × 300,000 tokens × $2.5/M = $900.00; 18,800 cache reads × 300,000 × $0.1/M = $564.00; 20,000 queries × (1,500 × $2/M + 400 × $10/M) = $140.00. Total $1,604.00.
  • RAG (Claude Sonnet 5.5, 7,900-token prompt): 20,000 queries × ((1,500 + 8 × 800) × $2/M + 400 × $10/M + 1,500 × $0.1/M embedding) = $399.00; vector store $70.00; re-indexing 4 × 300,000 × $0.1/M = $0.12; one-off (300,000 × $0.1/M + $4,000) ÷ 12 months = $333.34. Total $802.46.
  • Break-even: the volume where both totals match. Below 1,200 queries a month every query writes the cache, so long context costs $0.7570 a query; above it, $0.0370. RAG costs $403.46 a month before its first query, then $0.0200 a query.

Cost is not the only difference. Long context sees the whole corpus, so nothing is lost to a retrieval miss and questions that span sections work. RAG scales past the context window, keeps prompts short and fast, and cites the chunks it used. Claude API list prices, checked 2026-11-10; cache writes at the 5-minute rate. Embedding and hosting prices are estimates: change them to your provider's.

questions

how is prompt caching modelled?

The corpus is the cached prefix. A cache entry lives five minutes and every hit refreshes it, so while queries keep arriving the corpus is read at the cache-read rate. Each time traffic resumes after a pause of more than five minutes, the corpus is written again at the cache-write rate. Enter how many such pauses a day your traffic has; writes are capped at one per query.

what happens on haiku with a large corpus?

Haiku 5.5 bills prompts over 100K tokens at a higher long-context rate, for input, cache writes, cache reads and output. The calculator switches to that rate automatically when the corpus plus the query passes 100K tokens, and says so.

what if the corpus is bigger than the context window?

Then long context isn't an option: the corpus, the query and the answer must fit in one prompt. The calculator assumes a 1M-token window and tells you when the corpus doesn't fit. Splitting the corpus by topic and routing each question to one part is a middle path.

is cost the only thing to weigh?

No. Long context sees everything, so it can't miss a passage the retriever failed to find, and it handles questions that span sections. RAG keeps prompts short and fast, scales past any window, and cites the chunks it used. Measure answer quality on a golden set before choosing on price.

more free tools

Want the numbers checked against your own process? Book a 20-minute call.