A Private ChatGPT for Your Company Documents: What 'Private' Really Means

A private ChatGPT for your company documents is an AI assistant that answers from your internal files while keeping those files out of other people's reach: not used to train public models, not kept longer than you decide, and not readable by employees or apps who should not see them. You can get there with a business plan of a chat assistant, a self-hosted stack, or a private knowledge base that any assistant queries through an API or MCP. The right choice depends less on the brand of the chatbot than on three questions: who can read the documents, how long anything is kept, and where the answers come from.

What 'private' actually means for an AI assistant

'Private' is used loosely in AI marketing. When a vendor says your data stays private, it can mean very different things. Before you upload a single contract, split the word into the separate promises it can hide:

• No training use: your prompts and files are not used to improve a model that other customers will use.
• Limited retention: conversations, uploaded files and logs are deleted after a defined period, or when you delete them.
• Access control: only the people and applications you authorize can query the documents, and you can revoke that access.
• Isolation: your documents are not mixed into a shared index where another tenant's search could surface them.
• Contractual coverage: the promises above are written into terms you can point to, such as a data processing agreement or, for health data, a business associate agreement.

A setup can be strong on one promise and weak on another. A self-hosted model gives you excellent isolation but no access control unless you build it. A business chat plan may exclude your data from training but still let any colleague in the workspace open a shared file. Treat each promise as a separate checkbox.

Data retention: the questions to ask before uploading anything

Retention is where most surprises happen. Consumer and business tiers of the same assistant often have different defaults, and those defaults change over time. We will not summarize any vendor's current terms here, because they evolve: check the terms of your own plan, in writing, and keep a dated copy. These are the questions worth asking:

1. Are prompts, uploaded files and outputs used for model training by default, and can an administrator switch that off for the whole organization?
2. How long are conversations kept after a user deletes them, and are there backup or abuse-monitoring copies with a separate clock?
3. Are uploaded files stored as files, as extracted text, or as an index, and is each of those deleted when the source file is removed?
4. Who on the vendor side can access your content, under what conditions, and is that access logged?
5. Which subprocessors handle the data, and in which countries?
6. Can you export and delete everything at the end of the contract?

For US companies, the answers feed directly into your compliance work. If the documents contain protected health information, HIPAA requires a business associate agreement with any vendor that creates, receives, maintains or transmits that information on your behalf. If you sell to enterprise customers, their security questionnaires will ask whether your AI vendors hold a SOC 2 report. State privacy laws such as California's CCPA add notice and deletion obligations when personal information is involved. None of this forbids using AI on internal documents, but it does mean the retention answers need to be documented.

Four ways to chat with company documents privately

There is no single 'private ChatGPT' product. In practice, teams pick one of four patterns, sometimes two in parallel.

The fourth pattern is the least known and often the most practical. Instead of copying the same handbook into every chat tool your teams use, you keep the documents in one private retrieval layer and let assistants ask questions of it. That is retrieval-augmented generation, explained step by step in How RAG Works.

Access control: the part most teams skip

A private assistant that every employee can query about every document is not private inside the company. The HR investigation file, the board deck and the salary grid should not be one prompt away from an intern. Good access control for document chat follows the same principles as file sharing:

• Split by audience, not by topic. One base for the whole company (handbook, policies, product docs), one for HR, one for legal, one for finance. A question can only retrieve what the base contains.
• One key per application or person. If a script, an agent and a colleague all use the same credential, you cannot revoke one without breaking the others.
• Revoke on departure. Add API keys and assistant connections to your offboarding checklist, next to email and SSO.
• Keep personal data out unless it is needed. The safest sensitive document is the one you never indexed. Redact or summarize before uploading.
• Log questions where possible. You want to know which application asked what, especially for regulated content.

Private knowledge bases you query from ChatGPT, Claude or your own code

This is the approach Kopik takes. On Kopik, you create a knowledge base by uploading PDF, Word, text or Markdown files; Kopik extracts the text, splits it into passages and indexes it for hybrid search (full text plus semantic expansion of keywords). A private base is only accessible to its owner and to the owner's API keys. It never appears in the public catalog.

You can then query it three ways: on the website, through a REST API with a key, or through Kopik's MCP server, which MCP clients such as Claude, Cursor or ChatGPT can connect to (check what your workspace plan allows for custom connectors). Each answer is grounded in your documents and comes with numbered source passages, or you can ask for raw passages only and let your own assistant write the reply. The Model Context Protocol is an open standard, so the same base serves several assistants without being copied into each one.

For a single private base, the MCP address is `https://kopik.io/api/mcp?base=<slug>` and the key travels in the `Authorization: Bearer kpk_…` header. In Claude Code, for example:

`claude mcp add --transport http kopik "https://kopik.io/api/mcp?base=<slug>" --header "Authorization: Bearer kpk_…"`

The assistant then gets two tools: `ask_base` for a written answer with sources, and `search_base` for passages only. Content from the base arrives wrapped in `<kopik-untrusted>` tags, so the assistant treats it as data rather than as instructions, a useful protection if a document contains text that looks like a command. The full setup, including Cursor's `.cursor/mcp.json`, is in Connect a knowledge base to AI agents with MCP, and the reasoning behind keeping your own retrieval layer is in A private knowledge base as your own RAG for AI agents.

A rollout checklist for US teams

1. Inventory the documents. List what people actually ask about: policies, onboarding, product specs, contract templates, procedures.
2. Classify them. Public, internal, confidential, regulated (PHI, financial data, personal information under state law).
3. Pick the pattern per class. Internal handbooks can go into a private base quickly; regulated content may need a vendor that signs the agreements you require, or may stay out entirely.
4. Read the vendor terms. Training use, retention, subprocessors, deletion. Write the answers down with the date.
5. Create bases by audience and issue keys per app. Name keys after their use so revocation is obvious.
6. Test with real questions. Ask twenty questions you already know the answers to and check the cited passages, not just the wording.
7. Clean the corpus. Remove outdated versions; contradictory documents produce contradictory answers. Our guide on grounded answers covers the common causes.
8. Review quarterly. Keys, members, documents and vendor terms all drift.

Privacy for AI on documents is less about the chatbot and more about the plumbing: what is indexed, who holds a key, and how long anything is kept.