RAG with internal data: secure authorizations, sources and quality
By Nico Freitag, Geschäftsführer, Cybersecurity und Governance
Retrieval-Augmented Generation connects a language model with selected internal sources. The structure only becomes safe when document permissions apply until the answer is received, content can be versioned and deleted, sources remain visible, prompt injection is taken into account and the quality is measured by real questions. A vector database alone does not solve any of these problems.
Which sources of knowledge may be included?
Start with a limited inventory whose owner, protection needs and update path are known. Duplicates, outdated versions and unclear drafts not only make the answer worse, but also make accountability more difficult. Documents receive source, version, validity, technical owner and authorization features. Only then are sections created and indexed. A later answer must be able to be traced back to this metadata.
How do existing permissions remain effective?
The retrieval must not find any content that the requesting user should not see in the source system. For this purpose, identity and roles are transferred to the search and filtered before the model is transferred. Merely masking out the surface is not enough. Checks group changes, departures, confidential projects and approvals for external persons. Authorization tests are part of automated quality assurance and are repeated after changes to the index, search or role model.
How are prompt injection and manipulated sources handled?
Internal documents are not automatically trustworthy. An embedded text may attempt to override system rules, request data, or persuade the user to take a harmful action. Technically separates system instruction, user input, and retrieved content and treats sources as data, not commands. Restricts tools and privileges, validates spending, and requires explicit human decision-making on key actions. The BSI describes prompt injection as a relevant vulnerability when using large language models.
How do comprehensible answers come about?
An answer shows the sources used with title, section and, if possible, a direct link. If no sufficient basis has been found, the system should identify the uncertainty instead of filling gaps from general model knowledge. Determines which statement a source must actually support. Displaying sources is not proof of quality if the quoted section does not contain the answer. Therefore, the answer and the evidence are evaluated together.
How do updates and deletion work?
Defines events or fixed intervals for new, changed and deleted documents. A deletion must reach original, cache, generated sections, search index, logs and backups according to the specified rules. Incorrect imports require a visible quarantine path. For each index, record the source from which it was created. In the event of a faulty run, you can go back to a checked status.
Which metrics show real benefit?
Evaluates search hits, source coverage, technical accuracy, improper disclosures, response time, and necessary manual correction effort. Add user feedback to technical metrics, but don't confuse agreement with accuracy. A test set contains normal, difficult and intentionally misleading questions. For critical processes, there is a clear limit beyond which no automatic response or action is allowed. The company then specifically improves data, search or process instead of just changing the prompt.
From the answer to implementation
Related service
View the scope, delivery model and responsible contacts.
Open →Related product
See a practical product path connected to this topic.
Open →Working checklist
Prepare the next decision with a structured checklist.
Open →All specialist articles
Continue with reviewed answers from the same practice areas.
Open →Sources and basis
The central statements in this article were reviewed against the following primary sources.
Frequently asked questions
- Does every RAG system need a vector database?
- No. Depending on the data and question, classic searches, database queries or a combination may be more suitable. Reliable, authorized and traceable retrieval is crucial.
- Does RAG completely prevent invented answers?
- No. Good sources and clear response rules can reduce risk. Technical tests, visible evidence and reliable rejection remain necessary.
- Are sensitive documents allowed in a local RAG system?
- Only after a clear purpose, protection requirement, role model, logging and technical testing. Local does not automatically mean that every internal user can have access.