AI and LLM Glossary for Professionals: 11 Terms Explained Clearly
If you've sat through a meeting where someone said 'we need RAG, not fine-tuning, and make sure the agent uses MCP,' this glossary is for you. Below are plain-English definitions of the eleven AI and LLM terms that show up most often in product, engineering and compliance conversations, with enough technical detail to use them correctly. No marketing fluff, just working definitions you can repeat in a meeting.
Why This Glossary Matters for Business Professionals
AI vocabulary moves fast, and the same word often gets used loosely by vendors, journalists and engineers to mean three different things. Precision matters here: whether you're writing an RFP, evaluating a vendor's claim about retrieval augmented generation, or reviewing a data processing agreement that mentions embeddings, knowing exactly what each term covers (and does not cover) helps you ask better questions and avoid buying the wrong tool. This glossary groups terms into four practical clusters: the language model itself, how it finds and uses external knowledge, how its behavior is shaped, and how it connects to the tools you already use.
Core Language Model Terms
LLM (Large Language Model)
A large language model is a neural network trained on very large volumes of text to predict the next word (technically, the next token) given everything that came before it. That simple training objective, repeated across billions of examples, produces a model that can write, summarize, translate, answer questions and generate code. LLMs do not know facts the way a database does; they generate statistically plausible text based on patterns learned during training. That is why grounding an LLM's answers in real documents, the core idea behind RAG, matters so much for business use.
Token
A token is the basic unit an LLM reads and writes. It is usually a word, part of a word, or punctuation mark rather than a full sentence; in English, a rough rule of thumb is that 100 tokens equal about 75 words. Tokens matter commercially because most LLM providers price their APIs per token (input and output separately), and they matter technically because every model has a maximum number of tokens it can process at once, which brings us to the context window.
Context Window
The context window is the maximum amount of text (measured in tokens) a model can consider in a single request, including the prompt, any retrieved documents, and the conversation history. A model with a 128,000-token context window can technically read a few hundred pages at once, but stuffing the window full is not always the right strategy: it increases cost and latency, and retrieval quality often degrades when relevant information is buried among irrelevant text. The tradeoff between retrieving a few relevant passages versus pasting an entire document into the prompt is covered in detail in RAG vs Long-Context Models.
Retrieval and Knowledge Terms
Embedding
An embedding is a numeric representation of a piece of text (a word, sentence, or paragraph) as a list of numbers, typically a vector of several hundred or a few thousand dimensions. Text with similar meaning produces vectors that sit close together in that numeric space, which lets software compare meaning mathematically instead of just matching exact words. If you want the mechanics without the math, Embeddings Explained for Non-Specialists walks through how text becomes numbers step by step.
Vector Database
A vector database is a data store optimized for finding the embeddings that are mathematically closest to a given query vector, a task called similarity search. It is the component that makes retrieval fast at scale: instead of comparing a query against every document one by one, a vector database uses indexing structures to narrow millions of candidates down to the handful that matter in milliseconds. See What Is a Vector Database? for a deeper comparison of indexing approaches and when you actually need one.
RAG (Retrieval-Augmented Generation)
RAG is the pattern of retrieving relevant passages from a knowledge base before asking an LLM to answer, so the model's response is grounded in specific documents rather than its general training data. A well-built RAG system cites the passages it used, which lets a human verify the answer instead of trusting it blindly. The full mechanics, from chunking to indexing to generating a cited answer, are explained in How RAG Works, Step by Step, and What Is RAG? covers the concept from a business decision-maker's point of view.
Reranker
A reranker is a second, more precise model that re-sorts the initial batch of search results before they reach the LLM. Vector search and keyword search are fast but approximate; a reranker reads the actual query alongside each candidate passage and scores how relevant it really is, pushing the best matches to the top. This two-stage approach, fast retrieval followed by precise reranking, is one reason hybrid search consistently beats pure vector or pure keyword search for RAG.
Behavior and Customization Terms
Fine-tuning
Fine-tuning means continuing to train an existing LLM on a smaller, specific dataset so it adopts a particular style, format, or narrow skill, such as writing in a brand's voice or classifying support tickets. It changes the model's weights permanently, which makes it a poor fit for keeping up with frequently changing facts: every update to the underlying knowledge requires retraining. RAG solves a different problem, grounding answers in current documents, and the two are often confused; RAG vs Fine-Tuning breaks down which one actually fits a given business need.
Hallucination
A hallucination is a confident, fluent, and factually wrong statement produced by an LLM. It happens because the model is generating the most statistically likely next tokens, not consulting a verified source, so when it lacks real information it can still produce a plausible-sounding answer. Grounding answers in retrieved, cited passages is the most reliable mitigation available today; see How to Reduce AI Hallucinations With Grounded Answers and Document Q&A With Citations: How to Check an AI Answer for practical verification habits.
Agent and Integration Terms
Agent
An AI agent is an LLM-based system that can take multiple steps, call external tools such as a search engine, a database or a code runner, and decide what to do next based on the results, rather than producing a single one-shot answer. Agents are what turn an LLM from a text generator into something that can actually complete a task, like researching a topic across several sources or filing a support ticket.
MCP (Model Context Protocol)
MCP is an open standard that lets AI applications like Claude, Cursor or ChatGPT connect to external tools and data sources, including knowledge bases, through a common protocol instead of a custom integration for every pair of apps. A knowledge base exposed as an MCP server can be queried directly from inside an MCP client's chat window. Model Context Protocol (MCP) Explained covers servers, clients, tools and resources in depth, and How to Connect Claude, Cursor or ChatGPT to a Knowledge Base is the practical setup guide.
RAG vs Fine-Tuning at a Glance
| Dimension | RAG | Fine-tuning |
|---|---|---|
| What it changes | Which documents the model sees at query time | The model's internal weights |
| Best for | Fast-changing facts, citations, compliance answers | Style, tone, narrow classification tasks |
| Update cost | Add or edit documents, no retraining | Requires retraining on new data |
| Typical setup time | Hours (upload and index documents) | Days to weeks (dataset prep, training runs) |
Quick gut check
If your use case needs an answer traceable to a specific page in a specific document, such as a policy, a contract clause, or a regulation, you need retrieval, not fine-tuning. If it needs a model to sound a certain way or handle a repetitive formatting task, fine-tuning or even a good prompt is often enough.
- LLM: the engine that generates text
- Token: the unit it is billed and limited by
- Context window: how much it can read at once
- Embedding: the numeric fingerprint of meaning
- Vector database: where those fingerprints are searched
- RAG: the pattern that grounds answers in real documents
- Reranker: the step that sorts search results by true relevance
- Fine-tuning: retraining the model itself
- Hallucination: a confident wrong answer, mitigated by grounding
- Agent: a system that takes multi-step actions
- MCP: the protocol that connects agents to tools and data
Where to See These Concepts Working Together
Kopik (kopik.io) ties several of these terms into one workflow: upload documents and they get chunked, embedded and indexed for hybrid search; ask a question and get a cited answer instead of a hallucination; query from a catalogue of ready-to-use bases or build your own private one, and connect it to Claude or Cursor through MCP without writing a server yourself. If you want to see the REST API and MCP docs that make this possible, Kopik's developer documentation covers both.
Put these terms into practice
The fastest way to really understand RAG, embeddings and MCP is to build a small knowledge base and query it yourself.
How These Terms Relate to Each Other
It helps to see the pipeline end to end: documents are split into chunks, each chunk is turned into an embedding and stored in a vector database; a user's question is embedded the same way and matched against those vectors; a reranker sorts the matches by true relevance; the best passages and the question are sent to an LLM inside its context window, measured in tokens; the LLM generates a grounded answer instead of a hallucination. Wrap that whole flow in a system that can call tools and decide its next step, and you have an agent, which can expose or consume that pipeline through MCP. Chunking strategy alone has a measurable effect on answer quality, which is why it gets its own deep dive in Chunking Strategies for RAG.
Frequently asked questions
What is the difference between a vector database and a knowledge base?
A vector database is the storage and search engine for embeddings; a knowledge base is the higher-level product built on top of it, typically combining document storage, chunking, indexing (vector and often keyword) and an interface to ask questions. A vector database is one ingredient, not the finished product.
Is RAG the same thing as a chatbot?
No. RAG is the retrieve-then-generate pattern used inside a system; a chatbot is just the conversational interface. You can have RAG without a chat window, such as an API returning cited answers, and you can have a chatbot with no retrieval at all, just a plain LLM answering from memory.
Do I need to understand tokens to use an LLM API?
Only loosely. You mainly need to know that tokens drive both cost and the context window limit, so long documents or long conversations get truncated or billed more, not that you need to count them by hand.
Can an LLM hallucinate even when using RAG?
Yes, though less often. RAG reduces hallucination by grounding answers in retrieved passages, but a model can still misread a passage or blend it with unrelated training knowledge. That is why checking the cited source passage, not just trusting the answer, remains good practice.
What is the simplest way to explain MCP to a non-technical colleague?
MCP is like a universal plug: instead of building a custom wire between every AI app and every data source, tools and data expose themselves through one standard connector that any MCP-compatible app, such as Claude or Cursor, can plug into.
What is the difference between an LLM and an AI agent?
An LLM is the underlying text-generation model; an agent is a system built around one or more LLM calls that can take multiple steps, use tools, and decide what to do next. Every agent relies on an LLM, but not every LLM call is part of an agent.
Get the Kopik newsletter
New knowledge bases, RAG guides and product news. One email every week or two, unsubscribe in one click.
By subscribing you agree to receive our newsletter. We never share your address.