rag vs long context cost calculator
For questions over a fixed reference corpus, such as a policy manual, a tariff nomenclature or a contract library, there are two ways to give the model what it needs. Put the whole corpus in every prompt and let prompt caching absorb the repeat, or retrieve the few chunks that look relevant and pay for embeddings and a vector store. Put in your workload and see which costs less, where the two cross, and the arithmetic behind both.
rag is cheaper by $802 a month
the arithmetic
- Long context (Claude Sonnet 5.5, 301,500-token prompt): 1,200 cache writes (40 a day × 30, at most one per query) × 300,000 tokens × $2.5/M = $900.00; 18,800 cache reads × 300,000 × $0.1/M = $564.00; 20,000 queries × (1,500 × $2/M + 400 × $10/M) = $140.00. Total $1,604.00.
- RAG (Claude Sonnet 5.5, 7,900-token prompt): 20,000 queries × ((1,500 + 8 × 800) × $2/M + 400 × $10/M + 1,500 × $0.1/M embedding) = $399.00; vector store $70.00; re-indexing 4 × 300,000 × $0.1/M = $0.12; one-off (300,000 × $0.1/M + $4,000) ÷ 12 months = $333.34. Total $802.46.
- Break-even: the volume where both totals match. Below 1,200 queries a month every query writes the cache, so long context costs $0.7570 a query; above it, $0.0370. RAG costs $403.46 a month before its first query, then $0.0200 a query.
Cost is not the only difference. Long context sees the whole corpus, so nothing is lost to a retrieval miss and questions that span sections work. RAG scales past the context window, keeps prompts short and fast, and cites the chunks it used. Claude API list prices, checked 2026-11-10; cache writes at the 5-minute rate. Embedding and hosting prices are estimates: change them to your provider's.
questions
how is prompt caching modelled?
The corpus is the cached prefix. A cache entry lives five minutes and every hit refreshes it, so while queries keep arriving the corpus is read at the cache-read rate. Each time traffic resumes after a pause of more than five minutes, the corpus is written again at the cache-write rate. Enter how many such pauses a day your traffic has; writes are capped at one per query.
what happens on haiku with a large corpus?
Haiku 5.5 bills prompts over 100K tokens at a higher long-context rate, for input, cache writes, cache reads and output. The calculator switches to that rate automatically when the corpus plus the query passes 100K tokens, and says so.
what if the corpus is bigger than the context window?
Then long context isn't an option: the corpus, the query and the answer must fit in one prompt. The calculator assumes a 1M-token window and tells you when the corpus doesn't fit. Splitting the corpus by topic and routing each question to one part is a middle path.
is cost the only thing to weigh?
No. Long context sees everything, so it can't miss a passage the retriever failed to find, and it handles questions that span sections. RAG keeps prompts short and fast, scales past any window, and cites the chunks it used. Measure answer quality on a golden set before choosing on price.
more free tools
- ai roi & token cost estimatorMonthly Claude API cost, cost per task and ROI for a workload
- eu ai act compliance auditorRisk tier, obligations and dates for one AI system
- agent workflow fit scorerIs this process a good fit for AI agents, and what shape?
- ai build vs buy calculatorThree-year cost: SaaS tool, custom agents, or keep it manual
- ai accuracy proof calculatorHow many test cases prove a vendor's accuracy claim?
- ai model routing calculatorWhat sending each request to the cheapest capable model saves
- human review staffing calculatorHow many reviewers an AI workflow's escalations need
- ai vendor scorecardWeighted comparison of AI vendors, with must-haves and a swing check
- gdpr dpia screener for aiDoes this AI use of personal data need a DPIA?
- agent loop cost calculatorWhat a tool-calling agent costs per task as its context grows
- llm judge agreement calculatorDoes your LLM judge agree with people beyond chance? Cohen's kappa
- escalation threshold calculatorAt what confidence should an AI workflow hand a case to a person?
Want the numbers checked against your own process? Book a 20-minute call.