Guides · Owning your product
Guides · Owning your product
A RAG engine over your business data is an AI system that answers a question by first retrieving the relevant records from your own documents, then writing an answer grounded in what it found. RAG stands for retrieval-augmented generation. You give the model a private index of your contracts, policies or product data and let it read the right pieces at question time. This guide covers what a RAG engine does, when it beats fine-tuning, how per-tenant isolation keeps one client's data out of another's answers, and what it costs.
An AI engine over private data is the part of self-hosting most founders underestimate. If you are still deciding who runs your stack, start with the pillar on what self-hosted SaaS in production takes; this guide zooms into the engine at the centre of it.
A RAG engine has two halves: a retriever that finds the relevant chunks of your data, and a model that writes an answer from them. The retriever searches a private index built from your documents, and the model only sees the pieces it pulled back, plus the question.
The idea comes from the 2020 paper that named it: Lewis and colleagues paired a model's trained weights (parametric memory) with an external index it reads at query time (non-parametric memory), and found the pairing produced more specific and factual answers than the model alone (Lewis et al., 2020). Cloud vendors describe the same shape: the model references an authoritative knowledge base outside its training data before it answers, so it can cite a source and stay current without retraining (AWS).
Your business data lives in the index, not the model. Update a document, re-index it, and the next answer reflects the change.
RAG wins when the answer depends on facts that change or that must be traceable to a source. Fine-tuning wins when you need the model to learn a skill, format or tone it does not already have. They solve different problems, and the honest answer is often both.
The split is where the knowledge lives. Fine-tuning bakes it into the weights, so updating a fact means retraining; RAG keeps it in an index, so updating means editing a document. For business data that moves, prices, policies, inventory, case files, that decides it.
| Dimension | RAG over your data | Fine-tuning |
|---|---|---|
| What it changes | The data the model can read | The model's behaviour and style |
| Updating a fact | Re-index one document, minutes | Retrain the model |
| Source attribution | Yes, it can point at the record | No, the answer is baked in |
| Best fit | Facts that change or must be cited | A consistent skill, format or tone |
| Failure when wrong | Retrieves the wrong chunk | Learns the wrong pattern, harder to spot |
Most business questions are answered by retrieval alone. The hybrid, fine-tune once for voice and let RAG supply the live facts, is worth it only when you genuinely need both.
Per-tenant isolation means one client's data can never appear in another client's answer, and in a RAG engine that guarantee lives in the retriever, not the model. Every retrieval query has to be scoped to the tenant asking before a single chunk reaches the model.
The failure mode is quiet. Put every client's documents in one shared index with no tenant filter and it looks fine in a demo, then one day it retrieves a competitor's contract because that was the closest match by meaning. The model writes a clean answer from it. No error, just a leak.
We build isolation as a hard filter at retrieval time, one enforced tenant key per client, so a query can only see its own data. On one system we run, several LLM providers each cover a different feature with full data separation enforced in the retrieval and routing layer, not left to the model to respect. The holes that come from getting this wrong are the same ones in security holes we keep finding in AI-generated apps.
A RAG answer is trustworthy when it is grounded in a retrieved source and checked before anyone acts on it, not when it merely sounds right. Because the answer points back at the record it came from, a reviewer confirms the source in seconds, and an answer you can check beats a confident answer you cannot.
One system we run leans on this hard. A legal-document AI reads a client's own evidence database and drafts response letters in Hebrew, each grounded in that material and in the relevant religious law, and every draft passes several verification gates before a person approves it. The gates exist because a plausible wrong answer, in that setting, is expensive. The rule holds for any AI over business data: retrieve, ground, then verify the ground before you trust the output.
A RAG engine over your business data is part of the production-platform work we price by scope: ₪35K–60K for a full platform that includes the engine, per-client isolation, an admin area and backups. The model API calls are billed by usage on top, and not marked up.
Cost and latency come mostly from retrieval and the model call, not the index on disk. A retriever that returns a few relevant chunks keeps both down; one that stuffs everything into the prompt raises the bill and the wait for no gain, which is why the tuning is where a RAG build earns its price.
The model API is the metered part of a RAG engine, billed by usage. The retriever is where the cost is won or lost: return the three chunks that answer the question, not the thirty that might.
The full tier breakdown is on the pricing page under the Production Platform tier, and where a RAG engine fits the wider question of who owns and runs your stack is in self-hosted vs managed SaaS: an honest comparison.
A RAG engine stalls where retrieval stalls: it can only answer from what is in the index, and the answer is never better than the chunk it found. If the data is missing, messy or badly split, the model fills the gap with a guess, and grounding cannot rescue an answer that retrieved the wrong thing. It also will not teach the model a new skill: a different reasoning style, a strict output format or a domain voice the model lacks is fine-tuning's job, not retrieval's.
The honest test is one question: does the answer depend on your specific, changing data, and does someone need to trust it? If yes, RAG over your data is the right engine. If not, you are adding machinery for a problem you do not have.
If you want an AI engine that answers from your own data and keeps each client's records to themselves, get a written quote for a RAG engine over your business data. It is valid for 14 days, and it names what we build, what stays billed by usage, and what it costs.
Founder and main developer at Zyntari. Building apps and AI solutions since 2024; before that, 12 years in sales and client-portfolio management and co-founder of two companies. More about the team · LinkedIn