RAG for Accountants and Tax Advisers: Querying Guidance and Procedures With Sources
A tax season question rarely has one answer. It depends on the Internal Revenue Code section, the latest Treasury regulations, a Revenue Ruling that modified prior guidance, and often a state Department of Revenue position that contradicts federal treatment. Retrieval-augmented generation (RAG) lets a firm put all of that, plus its own internal procedures and client memos, into one searchable system that answers with the exact passage cited, instead of a confident-sounding guess.
Why tax research is a knowledge-retrieval problem
The Internal Revenue Code alone runs past 2,600 pages before you add Treasury regulations, IRS publications, Revenue Rulings, Revenue Procedures, private letter rulings and Tax Court decisions. Layer on 50 states each with their own conformity rules, apportionment formulas and filing deadlines, and the total volume of authoritative text a tax professional should theoretically be able to recall is far beyond what any memory, or any generic chatbot's training data, can reliably hold current.
Two things make this worse than a typical document search problem. First, the material changes every year: inflation adjustments, new elections, sunset provisions, and mid-year guidance from the IRS can flip an answer that was correct twelve months earlier. Second, tax terminology is dense and specific: a question about the Section 199A deduction, the at-risk rules under Section 465, or passive activity loss limitations under Section 469 needs a system that understands those exact terms, not just their general meaning. That is one reason plain keyword search in a shared drive fails here, and also why pure semantic search alone can blur distinct code sections together. For background on why combining both techniques matters, see Hybrid Search: Why BM25 + Vectors Beats Either Alone for RAG.
What RAG actually adds over a generic AI assistant
A general-purpose language model has been trained on a snapshot of public text and will happily answer a tax question from memory, with no way to tell you whether that memory reflects the current tax year, a repealed provision, or a state that no longer conforms to a particular federal rule. RAG changes the mechanism: instead of answering from memory, the system retrieves the most relevant passages from a defined set of documents you control (IRS publications, your firm's internal tax manual, prior client memos) and generates an answer grounded in those passages, with citations back to the source. If you haven't seen the mechanics end to end, How RAG Works, Step by Step walks through a single question from upload to cited answer.
This matters for three practical reasons in a tax practice:
- Verifiability: a staff accountant can click through to the exact paragraph of Rev. Proc. or the exact page of your firm's internal SALT procedures, rather than trusting an unsourced claim.
- Currency: when you replace last year's publications with this year's, the system answers from the new text immediately, no retraining required.
- Internal knowledge capture: a firm's unwritten rules ('we always file an extension for clients with K-1s from these three states') can be documented once and queried by anyone, including new hires during busy season.
Use cases that fit a tax and accounting practice
Federal and state guidance lookup
Upload the IRS publications, instructions and Revenue Procedures relevant to your practice areas (individual, pass-through, international, estate and gift) along with the state guidance for the jurisdictions where your clients file. A preparer can then ask something like 'what is the de minimis safe harbor election threshold this year' and get an answer tied to the specific publication and revision date, rather than relying on memory from a prior filing season.
Internal procedures and workpaper standards
Every firm accumulates internal memos on how it handles ambiguous situations: when to issue a 1099 for a borderline contractor relationship, how the firm documents reasonable compensation for S-corp owners, what the engagement letter template says about scope. These documents are rarely organized well and tend to live in someone's inbox. Turning them into a queryable base works the same way as Turn Your Internal Wiki Into an AI Assistant describes for other departments: upload what exists, let the system index it, and new staff stop interrupting partners with questions that are already answered in writing somewhere.
Client-specific research memos
Firms that produce technical memos for clients (nexus studies, entity structuring analyses, R&D credit documentation) can build a private knowledge base of past memos so that when a similar question comes up for a new client, the preparer can quickly check how the firm reasoned through a comparable fact pattern previously, with the original memo cited.
Keeping a tax knowledge base current year to year
Tax guidance has a predictable rhythm of change: inflation-adjusted figures each fall, new forms and instructions each January, mid-year IRS notices, and occasional retroactive legislation. A knowledge base that isn't maintained on that rhythm becomes actively dangerous, since a confidently cited answer from a superseded publication is worse than no answer at all.
Annual maintenance checklist for a tax knowledge base
| When | What to update | Why it matters |
|---|---|---|
| Before filing season opens | Replace prior-year IRS instructions and publications with current-year versions | Thresholds, standard deduction, and phase-outs change annually |
| After major legislation | Re-index affected code sections and add a summary memo of what changed | Avoids the system citing a repealed or amended provision as current law |
| Quarterly | Review and archive superseded Revenue Rulings and Notices | Keeps retrieval from surfacing guidance the IRS has already modified |
| Ongoing | Add new internal memos as positions are finalized | Captures institutional knowledge before staff turnover loses it |
Archive, don't delete, superseded guidance
Keep prior-year publications in a separate, clearly labeled base (or mark them as archived) rather than deleting them outright. Preparers sometimes need to answer a question about a prior tax year during an amended return or an IRS notice response, and having last year's rules retrievable, clearly dated, prevents confusion with current-year rules.
Building the system without a development team
A firm does not need an engineering team to set this up. On a platform like Kopik, creating a knowledge base starts with uploading the documents you already have: PDF publications downloaded from the IRS and your state's Department of Revenue, Word documents of internal procedures, and Markdown or text exports of past memos. The platform extracts the text, splits it into chunks sized for accurate retrieval, and indexes it for hybrid search that handles both exact statutory citations and plainer-language questions. You can start a base directly from the new base page.
Two deployment choices matter for a tax practice:
- Private bases keep documents visible only to the firm and its API keys, the right choice for internal procedures, client memos, and anything containing client-identifiable information.
- Public bases let a firm publish a paid knowledge base, for example a state-by-state nexus guide or a niche area like cost segregation, where other practitioners pay per question and the creator keeps 70% of the revenue. This is covered in more detail in Selling Your Expertise to AI Agents.
Once a base exists, staff can query it three ways: directly on the website during research, through a REST API if the firm wants to embed answers into an internal tool, or through an MCP server so that tools like Claude, Cursor or ChatGPT can pull cited answers straight into a workflow. Developers integrating this into firm software should start with the developer documentation.
Compliance and professional judgment still apply
RAG reduces the risk of a fabricated answer by grounding responses in real documents, but it does not turn a retrieval system into a substitute for a CPA's or EA's professional judgment, and it does not change the due diligence obligations under Treasury Department Circular 230. A cited passage still needs to be read in context, checked against the client's specific facts, and in many cases confirmed against the primary source before it goes into a filed return or a client-facing memo. Treat the system as a faster way to find the right passage, not as the final word.
Firms should also build a habit of checking the citations the system returns rather than trusting the generated summary alone. Document Q&A With Citations: How to Check an AI Answer covers the practical steps for that verification, and How to Evaluate a RAG System is useful if you want to measure how often the system's answers actually match what the cited passage says before rolling it out firm-wide.
Getting started this filing season
- Pick one practice area to start (individual, pass-through, or a single state) rather than trying to index everything at once.
- Gather the current-year IRS publications, relevant state guidance, and your firm's internal procedures for that area.
- Upload them into a private knowledge base and test it with the questions staff actually asked last season.
- Check a sample of answers against the cited source before trusting the system in production.
- Set a recurring reminder to refresh the base when new-year publications and forms are released.
Build a tax knowledge base your team can query with sources
Upload your firm's guidance and procedures, get hybrid search and cited answers, and query it from the website, the API or an MCP client like Claude or Cursor.
Frequently asked questions
Frequently asked questions
Can AI give tax advice to clients?
No. A RAG system can surface and cite the relevant guidance quickly, but the advice itself still needs to come from a licensed preparer who applies professional judgment to the client's specific facts, consistent with Circular 230 due diligence obligations.
How do I keep a tax knowledge base accurate after the IRS releases new guidance?
Replace superseded publications with current-year versions as soon as they're released, archive (rather than delete) prior-year documents so they remain available for amended returns, and add a short summary memo whenever legislation changes a code section the base covers.
Is it safe to upload client data into a knowledge base?
Keep any base containing client-identifiable information private, meaning only the firm and its API keys can query it. Public bases on marketplaces like Kopik are meant for general guidance and reference material, not client-specific records.
What is the difference between RAG and just asking ChatGPT a tax question?
A general assistant answers from training data that may be outdated or imprecise about which tax year or jurisdiction applies. RAG retrieves passages from documents you chose, current publications and your own procedures, and grounds the answer in those passages with a citation, which makes the answer checkable.
Can a small firm afford to build this without a developer?
Yes. Creating a knowledge base on a platform like Kopik only requires uploading the PDFs or Word documents you already have; the extraction, chunking and indexing happen automatically. A rough sense of ongoing costs is covered in What Does RAG Really Cost? A Full Cost Breakdown.
Can I query a tax knowledge base from Claude or another AI tool I already use?
Yes, through an MCP server. Once a knowledge base is set up, MCP-compatible clients such as Claude, Cursor or ChatGPT can query it directly as a tool, which is covered in How to Connect Claude, Cursor or ChatGPT to a Knowledge Base.
Get the Kopik newsletter
New knowledge bases, RAG guides and product news. One email every week or two, unsubscribe in one click.
By subscribing you agree to receive our newsletter. We never share your address.