RAG security: what changes when the AI can read your files

Retrieval-augmented generation is simple to describe. Instead of relying on what the model learned in training, the system finds relevant documents and puts them in front of the model as context. That is what makes an assistant able to answer from your own policies, contracts and files.

It is also a new application connecting a model to the company's information, and it needs to be designed like one.

The pipeline and its checkpoints

Documents, indexing, an embedding and vector store, retrieval, the model, the answer. Each stage is part of the security boundary, and each has its own question.

• Documents: who decided this set is in scope, and what happens when a document changes classification?
• Indexing: the index is a second copy of the content, so where does it live and who can read it?
• Vector store: embeddings are derived from the text, and OWASP lists vector and embedding weaknesses at LLM08
• Retrieval: are the user's own permissions applied here, or after the answer is written?
• Model: is retrieved content kept structurally distinct from instructions?
• Answer: are logs of the answer protected to the same standard as the source document?

The permission problem is the one that bites

Suppose the assistant can search everything the company holds. An employee asks for the chief executive's employment contract. The correct behaviour is not a refusal written into a prompt. It is that retrieval never returns a document the person could not open themselves.

That means RAG needs document-level access control enforced during retrieval. A shared index with a polite model on top is not access control; it is a search engine with an apology.

If the model has to be persuaded not to reveal a document, the permission was already wrong two stages earlier.

Every indexed document is untrusted input

Some of the documents in a corporate index were written by people outside the company. A supplier's invoice, a contract draft from the other side's lawyers, a candidate's CV, an inbound email.

That is the exact condition prompt injection needs, and it has produced a real vulnerability in a shipped product. CVE-2025-32711, disclosed in June 2025, was described as “a zero-click prompt injection vulnerability in Microsoft 365 Copilot that enabled remote, unauthenticated data exfiltration via a single crafted email”, and it scored 9.3 before Microsoft fixed it.

Local retrieval, and what it does and does not fix

A private deployment can keep the documents, the embeddings, the retrieval step and the inference inside the organisation. For sensitive material that is a genuine advantage, and it also simplifies the question of where the second copy lives.

It is worth noting that a documented cloud service can also avoid duplicating the store. Microsoft's documentation of its "on your data" feature states that “The data remains stored in the data source and location you designate; Azure OpenAI does not create a duplicate data store.”

So the local advantage is not that the cloud copies your files. It is that the permission model, the index and the network path are all under one administration rather than three.

Seven questions before go-live

• Who can access each document today, outside the AI system?
• Are those permissions enforced during retrieval, for every query?
• Can a user obtain, through the assistant, content they could not open directly?
• Which indexed documents were written by someone outside the organisation?
• Where do the embeddings live, and who can read them?
• Are prompt and answer logs protected like the documents they quote?
• What tools can the model reach, and what is the worst a misled model could do with them?

RAG does not make the AI "know your company". It builds a new application on top of your document permissions, and it inherits every flaw in them.