A large language model can explain a concept, draft a mail, or summarise a document you paste in. It cannot reliably know your price list, your machine manuals, your HR policy, or what changed in the plant last Tuesday. Those facts live in your files and systems, and they change.
Retrieval-augmented generation, usually called RAG, is the pattern that connects the two. The system searches your knowledge first, passes the relevant passages to the model, and asks the model to answer from that material. The model supplies language. Your documents supply the facts.
What an LLM Does, and What It Does Not
An LLM is trained to continue text in a convincing way. It is good at wording, structure, and general knowledge. It is a poor system of record.
- It can draft from instructions and from text you give it in the prompt.
- It can be wrong with confidence when the fact was not in the prompt and was not stable in its training.
- It does not update itself when you change a policy PDF on Monday morning.
- It should not be the only place a price, a safety step, or a legal clause is stored.
That is why asking the model alone fails for company knowledge, and why RAG became the usual way to put an LLM to work on private information.
How RAG Works, in Plain Steps
- Collect the sources: policies, manuals, product sheets, tickets, SOPs, and approved web pages.
- Break them into passages: short chunks that can be searched, with a title and a link back to the original.
- Index them: a search index, often with embeddings, so a question can find related passages even when the wording differs.
- Retrieve: when someone asks a question, fetch the few passages most likely to contain the answer.
- Generate: the LLM writes the answer using those passages, and says when the passages do not contain the answer.
- Show the source: the user can open the document and check the line the answer came from.
If you remove the last step, you have a fluent system that is hard to trust. Citations are not a decoration. They are how a manager, an engineer, or an auditor checks the work.
RAG, Fine-Tuning, and a Longer Prompt
Teams often mix these up. They solve different problems.
- A longer prompt: you paste the relevant text yourself. Fine for one document. It does not scale to a library, and it breaks when the text no longer fits.
- RAG: the system finds the current passages for each question. Best when the facts change, live in many documents, and must be traceable.
- Fine-tuning: you train the model on examples so it follows a format, a tone, or a specialised task. It does not reliably store your latest price list, and you cannot point to a paragraph when it is wrong.
For most company assistants, start with RAG. Consider fine-tuning later if you need a strict output format or a task the base model keeps doing poorly, and even then keep retrieval for the facts.
Where RAG Is a Good Fit
- Employee help desk: leave policy, IT steps, expense rules, and who approves a request.
- Customer and dealer support: product specs, warranty terms, and troubleshooting that must match the manual.
- Sales enablement: current offers, comparison sheets, and answers a representative can check before sending.
- Plant and field operations: SOPs, safety steps, and equipment manuals, answered in the language your team uses.
- Proposal and compliance drafts: pull the approved clauses and past answers, then let a person edit.
What Usually Breaks in the First Version
- Bad documents in, bad answers out: duplicate policies, untitled scans, and outdated PDFs will be retrieved as if they were current.
- Chunks that split the answer: a table cut in half, or a step separated from its warning, makes the model guess.
- Search that matches words, not the question: "reset the line" and "restart the machine" may be the same job in your plant and different jobs in the index.
- No refusal: if the instruction says to always answer, the model will fill gaps. Tell it to say when the sources are silent.
- No access control: a shared assistant that can read HR files and customer contracts will show the wrong passage to the wrong person.
- A stale index: if the search copy is updated once a quarter, the assistant will quote last quarter's process.
How to Judge Quality Before You Launch
Collect 30 to 50 real questions from the people who will use it. Include questions the documents cannot answer. For each one, record:
- Did it retrieve the right passage?
- Did the answer stay inside that passage?
- Did it refuse when it should have?
- Could a user open the source in one click?
Fix retrieval before you blame the model. In most first projects, a better document set and a clearer chunking rule improve answers more than switching to a larger LLM.
A Practical Build Order
- Choose one knowledge set that is owned, current, and allowed to be shown to the users you have in mind.
- Clean the obvious problems: remove duplicates, mark the latest version, and skip scans you cannot read.
- Index and test search on real questions before you connect a model.
- Add the LLM with strict instructions: answer only from the passages, quote the source, and say when it is not there.
- Put it in front of a small group and read the failures every week.
- Only then widen access, with permissions that match who may see each document.
Cost, Privacy, and the Choice of Model
- The model is only part of the cost. Ingestion, search, storage, and the time to clean documents are the rest.
- Private data should stay private. Decide what may be sent to an external model API and what must stay in your environment.
- A larger model is not automatically better. If retrieval is precise, a smaller model often writes a perfectly good answer at a lower cost and latency.
- Log questions with care. Keep enough history to improve the system, and no more than your policy allows.
How i10AI Solutions Builds RAG for a Business
We start from the documents and the questions, not from a model catalogue:
- Use-case and source review: who will ask, which documents are allowed, and what a wrong answer would cost.
- A searchable knowledge set: cleaning, chunking, and an index you can update when a document changes.
- An assistant with citations: answers tied to the passage, with a refusal when the source is missing.
- A test on your questions: we score retrieval and answers before a wider rollout.
- Handover: your team can add documents, see failures, and keep the index current.
An LLM becomes useful in a company when it can show where the answer came from. RAG is how you get there without pretending the model memorised your business.