Internal knowledge systems and source-grounded answers

An employee asks, "Can this customer return a custom order after 30 days?" A general language model can produce a fluent answer. The business needs something stricter: find the current approved policy, respect the employee's access, answer from that evidence and show where the answer came from.

That is the promise of a source-grounded knowledge system. It is useful, but it is not a guarantee of truth.

What retrieval-augmented generation does

Retrieval-augmented generation, usually shortened to RAG, separates two jobs:

1. retrieve passages likely to be relevant from an approved knowledge collection;
2. give those passages to a language model as context for an answer.

The original 2020 RAG paper combined model memory with retrieved external documents and found benefits on knowledge-intensive tasks. Lewis and colleagues

Microsoft's current overview describes the pipeline as retrieve, augment and generate, and highlights citations as a way to make answers auditable. Microsoft Learn

The hard part begins before retrieval

If the knowledge collection contains three contradictory policies, the system does not know which one represents company policy unless governance makes that clear.

Each source needs:

• owner;
• approval status;
• effective and expiry dates;
• audience and access classification;
• canonical location;
• revision history;
• review schedule.

Delete or label superseded material. A beautifully indexed archive of obsolete documents can produce beautifully cited obsolete answers.

Retrieval quality is measurable

Build a test set of real questions with expected sources. Include paraphrases, ambiguous terms, questions with no answer, and cases where access differs by user.

Measure whether the system retrieves the authoritative passage, not merely a semantically similar passage. Then evaluate whether the generated answer is supported by that passage.

Useful evaluation categories include:

• retrieval relevance: were the right passages returned?
• groundedness: is each factual claim supported by them?
• completeness: did the answer omit a material condition?
• citation correctness: does the cited location contain the claimed support?
• abstention: does the system decline when the sources are insufficient?
• access control: can a user retrieve only what they are allowed to see?

Citations are evidence, not decoration

A citation should open the exact document or section, show its status and let the reader inspect context. Linking every answer to a generic knowledge homepage is not traceability.

Require the model to distinguish source facts from inference. Where sources conflict, show the conflict or route it to the content owner. Where there is no approved source, say so.

RAG reduces reliance on model memory, but the model can still misread a passage, combine unrelated text or make an unsupported leap. The retrieval system can also miss the right document.

Permissions must travel with the content

An internal search assistant can become a new path around existing permissions. The index, retrieval filters, cache and answer should enforce document-level or finer access.

Microsoft's RAG guidance includes document-level access control as part of enterprise retrieval. OWASP separately identifies weaknesses in vector and embedding systems, including unauthorized access and cross-context leakage, and recommends permission-aware controls. Microsoft Learn and OWASP

Treat retrieved content as untrusted input too. A malicious or compromised document can contain instructions designed to manipulate the model. OWASP lists prompt injection as a major language-model risk and notes that RAG does not fully remove it. OWASP

Start with a narrow corpus

Choose one useful domain with named owners, such as approved HR policies or technical service procedures. Clean the sources, define permissions and collect real questions. Launch to a small group with a visible feedback route.

Do not begin by indexing every file the company has ever stored. More documents can make retrieval worse when they add duplication, stale versions and inaccessible material.

Decide what happens after the answer

A knowledge assistant should state what it can and cannot decide. It may summarize a policy but not approve an exception. High-consequence questions should route to a named role. Log enough to investigate quality while respecting privacy and retention rules.

The goal is not a bot that always answers. It is a system that helps people find defensible answers, shows its evidence and recognizes when the evidence is not enough.

Source check

RAG research and product guidance support retrieval-grounded architecture, but no source claims it eliminates hallucinations. Product-specific permission controls must be tested in the organization's configuration.

Sources

• Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Microsoft Learn: Retrieval-augmented generation
• Microsoft Learn: RAG in Azure AI Search
• OWASP: Prompt Injection
• OWASP: Vector and Embedding Weaknesses
• NIST: Generative AI Profile