What Does RAG Really Cost? Ingestion, Storage, Retrieval and Generation Explained

A single RAG query typically costs somewhere between a fraction of a penny and a few pence, depending on how many tokens you retrieve and generate, which embedding and generation models you use, and whether you pay per question or via a subscription. The real cost of RAG has four components: ingesting and indexing your documents, storing the resulting vectors, retrieving relevant passages for each question, and generating the final answer. Most of the ongoing spend sits in generation rather than retrieval, which is why chunking choices and prompt length matter more than people expect. Below is a full breakdown with worked numbers, plus practical ways to bring your bill down.

The four stages you're actually paying for

"RAG cost" gets quoted as a single number far too often, usually the price per 1,000 LLM tokens. That number ignores most of what actually happens when someone asks a question against a knowledge base. There are four distinct cost stages, and they behave very differently as your document set and query volume grow.

1. Ingestion: extracting text from PDFs, Word files or Markdown, splitting it into chunks, and running each chunk through an embedding model. This is a one-off cost per document (or per update).
2. Storage: keeping the vectors and metadata indexed somewhere queryable, month after month, whether or not anyone asks a question.
3. Retrieval: running a search (vector, keyword, or hybrid) across the index for each incoming question to find the right passages.
4. Generation: sending the retrieved passages plus the question to a language model and paying for the input and output tokens of that call.

Ingestion and storage scale with the size of your knowledge base. Retrieval and generation scale with the number of questions asked. Confusing the two is how teams end up surprised by their bill: a knowledge base with 10,000 documents and no traffic costs very little per month, while a small knowledge base answering 50,000 questions a day can rack up a substantial generation bill even though storage is negligible.

Where the money actually goes, stage by stage

Of these four, generation is usually the dominant recurring cost once a system is in production. A typical answer might retrieve 3 to 8 passages of 300 to 500 tokens each, so the input side of the generation call alone can run to 1,500 to 4,000 tokens before the model has written a single word of the answer. Our guide on chunking strategies for RAG covers how chunk size affects both retrieval quality and this exact input-token bill: smaller chunks retrieved in higher numbers can cost more in tokens than fewer, larger chunks, even though they often improve precision.

A worked example: what one question actually costs

Take a mid-sized knowledge base, say a 200-page employee handbook or a product manual. Ingestion (extraction plus chunking plus embedding) for that volume is typically a one-off cost measured in pence to a couple of pounds, done once and then only repeated when documents change. Storage for a few thousand chunks is small enough to be rounded to nothing in most providers' pricing.

The per-question cost is where attention should go. A hybrid search retrieval step (combining full-text matching with semantic keyword expansion, as described in our piece on hybrid search) adds a small, fairly fixed cost regardless of model choice. The generation step is the variable: a shorter, well-scoped context plus a concise answer might use 2,000 to 3,000 tokens total, while a verbose prompt with many retrieved passages and a long generated answer can use 6,000 tokens or more. Multiply that token count by your chosen model's per-token rate and you get your per-question generation cost, which is the number that should appear on any honest RAG pricing page.

Per-question pricing vs a flat subscription

Two commercial models dominate RAG and AI pricing generally: pay-per-question (prepaid credits, consumed as questions are asked) and a flat monthly subscription with a usage allowance. Each suits a different usage pattern.

• Pay-per-question suits spiky or low-volume usage: you only pay when someone actually asks something, so a knowledge base that sits idle for weeks costs nothing in that period.
• Subscriptions suit predictable, steady usage: a fixed monthly fee with a usage gauge is easier to forecast and often cheaper once you cross a certain volume threshold.
• Hybrid setups, where light users stay on pay-per-question and heavy users move to a subscription, let you match the pricing model to the actual traffic rather than guessing upfront.

On Kopik, this plays out directly: you can query knowledge bases with prepaid credits consumed per question, or take a chat subscription at €12 a month with a usage gauge if you ask questions regularly. For anyone publishing a public knowledge base, Kopik also handles the commercial side: the creator sets the price per question and keeps 70% of what's charged, so the economics of running a paid knowledge base are visible from day one rather than buried in a generic API bill.

Nine ways to bring your RAG bill down

1. Right-size your chunks. Smaller chunks retrieved in large numbers inflate input tokens; see chunking strategies for RAG for sentence-aware and structure-aware approaches that reduce noise.
2. Cap the number of retrieved passages. Four well-chosen passages usually beat ten mediocre ones, and halve your generation input cost.
3. Use hybrid search instead of pure vector search for exact terms (part numbers, clause references, dates). It reduces irrelevant retrievals that would otherwise pad the prompt, as explained in our hybrid search guide.
4. Don't over-provision context. For short factual answers, a tighter system prompt and a lower max-output-tokens setting directly cuts the output side of the bill.
5. Compare retrieval against long-context stuffing for your specific use case. Sometimes retrieval is cheaper, sometimes it isn't; our comparison of RAG vs long-context models walks through when each wins on cost.
6. Re-index only what changed. Diff your documents before re-ingesting the whole set; most knowledge bases change a handful of pages at a time, not the whole corpus.
7. Choose a usage model that matches your traffic shape: pay-per-question for occasional use, a subscription once volume is steady and predictable.
8. Separate "answer with citations" from "passages only" modes where you don't need a generated sentence, raw cited passages are cheaper because they skip the generation call entirely.
9. Monitor faithfulness, not just cost. A cheap answer that hallucinates and needs a human to redo the work costs more overall; see our guide on evaluating RAG systems for the metrics worth tracking alongside spend.

Self-host or use RAG as a service: the cost trade-off

Running your own stack (a vector database, an embedding pipeline, orchestration code and an LLM API key) gives you full control, but the fixed costs start before you've answered a single question: hosting, maintenance, and the engineering time to keep chunking, indexing and retrieval tuned. Our breakdown of vector database cost covers what you're really paying for there, and RAG as a service explains when outsourcing the plumbing is the better call.

A marketplace model like Kopik removes most of the fixed cost: upload your documents to a free knowledge base via the dashboard, and extraction, chunking and hybrid indexing are handled for you. You only pay (or your users pay) per question, or via the subscription, with no servers to size and no vector index to maintain. For teams that want to query programmatically, the developer docs cover the REST API and MCP server options, so the same cost model applies whether you're querying from a script, a chat subscription, or an MCP client like Cursor or ChatGPT.

A quick checklist before you sign off a RAG budget

• Have you separated one-off ingestion cost from recurring retrieval and generation cost in your estimate?
• Have you tested average token counts on real questions, not a hypothetical short query?
• Does your chunking strategy retrieve the minimum passages needed for a faithful answer?
• Have you compared pay-per-question against a subscription at your expected monthly volume?
• Do you have a plan (and a cost line) for re-ingesting documents when they change?

Frequently asked questions

A few questions that come up repeatedly when organisations budget for RAG and AI pricing more broadly.