What Does RAG Really Cost? A Full Cost Breakdown

Retrieval-augmented generation has four distinct cost buckets: ingestion, storage, retrieval and generation. For most business knowledge bases, the generation step, the single LLM call that writes the final answer, accounts for the largest share of the per-question cost. The database work that finds the right passages is usually cheap by comparison. Understanding this split is the first step to budgeting a RAG project correctly and to knowing which lever to pull when costs climb.

The four cost buckets in any RAG system

Every retrieval-augmented pipeline, whether you build it yourself or use a hosted service, moves money through the same four stages. Knowing them separately matters because each one scales differently: some costs happen once per document, others happen once per question, and one scales with how long the answer is.

• Ingestion: extracting text from PDFs, Word files or scanned pages, splitting it into chunks, and generating embeddings for each chunk. This is a one-time cost per document (unless the document changes).
• Storage: keeping the chunks, their embeddings and the full-text index available for search, month after month, whether or not anyone asks a question.
• Retrieval: the search itself, running a semantic or hybrid query against the index to find the handful of relevant passages. This happens once per question and is typically the cheapest step.
• Generation: sending the retrieved passages plus the question to a language model and paying for the tokens it reads and writes. This also happens once per question but usually costs far more than retrieval.

If you want the mechanics behind these four stages in more detail, including how chunking choices affect both ingestion cost and answer quality, see Chunking, indexing, retrieval: how a RAG pipeline works under the hood and What Is RAG (Retrieval-Augmented Generation)? A Practical Guide.

Ingestion costs: what it takes to get documents ready

Ingestion is a fixed, one-time cost that depends on document volume, not on how many people later ask questions. It typically includes text extraction (harder and slightly more expensive for scanned PDFs that need OCR than for clean text or Word files), chunking, and embedding generation. Embedding a chunk of a few hundred words is inexpensive, usually a small fraction of a cent, but a knowledge base with thousands of pages can add up to a meaningful one-time bill before the first question is ever answered.

The chunking strategy you choose also affects ingestion cost indirectly: smaller, more numerous chunks mean more embedding calls and a larger index, while larger chunks mean fewer calls but risk diluting relevance. Chunking Strategies for RAG: Fixed-Size, Semantic, and Structure-Aware Splitting Compared walks through the trade-off in detail.

Storage costs: the part that runs even when nobody asks a question

Storage is the only cost bucket that accrues on a calendar basis rather than per question. A vector database or hybrid search index needs to keep embeddings, metadata and often a full-text index resident and queryable, which costs money whether traffic is high or zero that month. For a small knowledge base (a handbook, a product manual, a set of legal memos), storage is usually negligible. It becomes a real line item once you are indexing millions of chunks across many large documents, or when you pay for a managed vector database with per-GB and per-read-unit pricing on top of compute.

If you are evaluating vector database options directly, Vector Database Pricing: What You Actually Pay For breaks down the different billing models (storage, query units, pod-based pricing) in more depth, and Pinecone Alternatives for Small Teams: pgvector, Open-Source Engines and Hosted Knowledge Bases compares self-hosted and managed options for teams who want to avoid building this layer from scratch.

Retrieval costs: usually the cheap part

Retrieval, the act of searching the index for the passages most relevant to a question, is almost always the least expensive step per question. A well-indexed hybrid search (combining keyword matching like BM25 with semantic vector similarity) typically costs a small, predictable amount per query, largely independent of how long the document corpus is, because the index is built to make lookups fast regardless of scale. Hybrid search also tends to produce better results than pure vector search alone, which matters for cost indirectly: better retrieval means fewer irrelevant passages sent to the language model, which keeps the next step (generation) cheaper too. See Hybrid Search: Why BM25 + Vectors Beats Either Alone for RAG for why this combination tends to win on both accuracy and cost.

Generation costs: where the money actually goes

Generation is the step most people mean when they ask about LLM cost per query. A language model call costs money based on tokens: the tokens it reads (your question plus every retrieved passage you stuff into the prompt) and the tokens it writes (the answer). This is why the number and length of passages you retrieve matters so much for cost. Retrieving ten long passages when three would answer the question just as well multiplies your input token cost for no quality gain.

Three factors drive generation cost in practice: which model you use (larger, more capable models cost more per token but may need fewer retries or follow-up questions), how many passages you pass into the prompt, and how long the generated answer is allowed to be. A well-tuned RAG system that retrieves tightly relevant chunks and asks for a focused answer with citations can keep generation cost far lower than a system that dumps an entire document section into the prompt "just in case."

Per-question pricing vs. subscription: which fits your usage

Once you know where the cost actually comes from, the pricing model you choose should match how predictable your usage is. There are two common approaches in the market, including on Kopik:

• Per-question (prepaid credits): you buy credits and spend a small amount each time a question is answered. This fits irregular or low-volume usage, internal tools used by a handful of employees, or a public knowledge base where you want costs to track revenue directly.
• Flat subscription with a usage gauge (for example, a chat plan at a fixed monthly price with a built-in question allowance): this fits steady, predictable usage, a support team or an internal assistant answering dozens of questions a day, where a flat fee is easier to budget than metered billing.

If you are building a knowledge base that other people or AI agents will query and pay for, the economics flip: you are no longer just paying for generation, you are setting a price per question and keeping a share of it. Selling Your Expertise to AI Agents: How to Build a Paid Knowledge Base covers how creators price public bases on Kopik, where the creator keeps 70% of what buyers pay per question.

Seven ways to lower your RAG costs

Most cost optimization in RAG comes down to spending less on generation without hurting answer quality, since that is usually where the bulk of per-question spend sits.

1. Retrieve fewer, better passages. Tightening your retrieval (hybrid search, better chunk boundaries) means you need three well-chosen passages instead of ten loosely relevant ones.
2. Cap answer length. Ask the model for a concise, cited answer rather than an open-ended essay; shorter outputs cost less and are usually easier to verify.
3. Right-size your chunks. Chunks that are too small force the model to see fragments without context (more retrieval round-trips); chunks that are too large waste tokens on irrelevant text. See Chunking Strategies for RAG.
4. Cache repeated questions. If the same or very similar questions come up often (a handbook FAQ, a product spec), caching the answer avoids paying for generation twice.
5. Use "passages" mode when you don't need a generated answer. If a user or an agent just needs the sourced text, skipping the generation step entirely removes its cost altogether.
6. Re-evaluate retrieval quality periodically. Poor retrieval leads to follow-up questions and retries, which multiply generation calls. How to Evaluate a RAG System: Retrieval Metrics, Faithfulness and Citation Accuracy covers how to measure this.
7. Compare RAG against stuffing everything into a long-context prompt. For small, stable document sets, a long-context model without retrieval can sometimes be cheaper or simpler; for large or frequently updated corpora, retrieval wins. RAG vs Long-Context Models: When to Stuff the Prompt and When to Retrieve lays out the trade-off.

Build vs. managed: where Kopik changes the math

Building your own RAG pipeline means paying separately for embedding calls, a vector database subscription, compute to run retrieval, and your LLM provider bill, plus the engineering time to wire all four together and keep them working as documents change. A managed marketplace like Kopik collapses ingestion, storage, retrieval and generation into one priced unit: you upload documents to a knowledge base through /dashboard/bases/new, and from then on you pay per question (prepaid credits) or a flat monthly subscription with a usage gauge, without separately tracking embedding bills or database invoices. For teams that just need reliable, cited answers over their own documents without standing up infrastructure, that bundling is often the actual cost saving, not any single optimization. For a broader comparison of managed RAG options, see RAG as a Service Platforms Compared: Managed Pipelines, Vector Stacks and Knowledge-Base Marketplaces and RAG as a Service: What It Is and When to Use It.

A quick budgeting checklist

Before committing to a pricing model or a build-vs-buy decision, it helps to estimate these numbers on paper first:

• How many documents and total pages will you ingest, and how often do they change (one-time ingestion cost vs. recurring re-indexing)?
• How many questions per month do you realistically expect, and how steady is that volume (favors subscription) or how spiky is it (favors per-question credits)?
• What is an acceptable answer length for your use case? Shorter, cited answers are both cheaper and easier for a human to verify.
• Do you need a generated answer at all, or would cited raw passages (passages mode) satisfy most questions?
• If you plan to monetize a public knowledge base, what price per question lets you recover generation cost and still leave margin after the platform's revenue share?

If you want to query a knowledge base programmatically once you have an estimate in hand, whether to compare provider pricing or to wire a chatbot into your own product, the /developers documentation covers the REST API and the MCP server for connecting tools like Claude, Cursor or ChatGPT directly to a base.