Retrieval vs generation spend
Feeding retrieved chunks into the model (input tokens) is usually the bigger line in RAG.
Home / Dev & AI / AI & LLM Cost / RAG Token Cost Calculator
AI & LLM Cost
RAG Token Cost Calculator
See RAG costs at the token level. Set context tokens per query, answer length and models, and this RAG token cost calculator shows the retrieval versus generation split and the cost per query.
Estimated monthly cost
$0
One-time index cost—
Cost per query—
Answer generation / month—
Query embedding / month—
Queries per month—
Annual query estimate—
Your RAG answers priced across LLMs
Same query load, cheapest answer model first.
What the RAG Token Cost Calculator does
The RAG token cost calculator zooms into the tokens that make retrieval augmented generation cost what it does. Each query carries three token flows: the small embedding of the question, the retrieved context tokens you pass to the model, and the answer tokens the model writes. The context flow is usually the largest, since good grounding means passing several relevant chunks every time.
By pricing each flow separately, the tool shows exactly where to optimise. If retrieval tokens dominate, tighten your chunking and top-k. If answer tokens dominate, cap response length. Either way you can compare answer models on the same token profile to find the cheapest that still gives good answers.
Heads up on pricing. The rates built into this tool are public list estimates for July 2026 and can change without notice. Providers also offer batch discounts, prompt caching and volume tiers that lower real costs, so treat the output as a planning estimate, not a quote.
The formula
context cost = context tokens ÷ 1M × input rate
answer cost = answer tokens ÷ 1M × output rate
per query = query embed + context cost + answer cost
answer cost = answer tokens ÷ 1M × output rate
per query = query embed + context cost + answer cost
Worked example
A query passing 3,000 context tokens and writing 400 answer tokens on Gemini 2.5 Flash costs 0.003 × $0.30 + 0.0004 × $2.50 = $0.0009 + $0.001 = about $0.0019 per query. At 100,000 queries a month that is roughly $190.
AI model pricing used in this calculator (July 2026 estimates)
| Model | Provider | Input / 1M tokens | Output / 1M tokens |
|---|---|---|---|
| GPT-5 | OpenAI | $1.25 | $10.00 |
| GPT-5 mini | OpenAI | $0.25 | $2.00 |
| Claude Opus 4.1 | Anthropic | $15.00 | $75.00 |
| Claude Sonnet 4.5 | Anthropic | $3.00 | $15.00 |
| Claude Haiku 4.5 | Anthropic | $1.00 | $5.00 |
| Gemini 2.5 Pro | $1.25 | $10.00 | |
| Gemini 2.5 Flash | $0.30 | $2.50 | |
| DeepSeek V3 | DeepSeek | $0.27 | $1.10 |
| Grok 4 | xAI | $3.00 | $15.00 |
| Llama 4 Maverick | Meta | $0.20 | $0.60 |
Prices are public list estimates for planning as of July 2026 and change often. Providers bill per token, where roughly 1,000 tokens equals about 750 words. Always confirm live rates on the provider pricing page before you set a budget.
Ways to lower your AI costs
- Match the model to the task. Route simple requests to a small model like GPT-5 mini, Gemini Flash or DeepSeek V3, and keep premium models for complex reasoning where quality pays off.
- Cache repeated context. Prompt caching means you stop paying full price for the same system prompt or documents on every call.
- Cap output length. Output tokens cost several times more than input, so limiting response length is often the fastest saving you can make.
- Trim prompts and history. Summarize old conversation turns and drop context the model does not actually need to answer well.
- Batch non-urgent work. Many providers offer batch pricing at around half the standard rate for jobs that can wait a few hours.
Frequently asked questions
Which tokens cost the most in RAG?
Usually the retrieved context tokens, because you resend several chunks as input on every query. Reducing how many chunks you pass has the biggest cost impact.
How many context tokens should I pass?
Enough to answer well and no more. Many pipelines pass 2,000 to 4,000 tokens of top ranked chunks. Passing your entire corpus is both slow and expensive.
Does a bigger answer cost more?
Yes, answer tokens are output tokens, the priciest kind. Cap the maximum length to keep per query cost predictable.
Can caching help RAG?
Yes. Caching a stable system prompt and instructions avoids re-billing those tokens on every query, which adds up across high volume.
Are the token rates current?
They are July 2026 estimates. Verify the live per million token rate with the provider before planning.