RAG Deployments Are Leaking Privileged Data Through a Gap Evaluations Cannot See
The Development
Egiziago Cioffi, CEO of Microsoft partner SynSphere Italia, built a custom Azure OpenAI email assistant that auto-resolves roughly 60% of inbound customer email. After the agent passed every internal evaluation, Cioffi ran a low-privilege account against questions a high-privilege account had already posed. The assistant returned SharePoint content the low-privilege user could not have accessed directly. The failure is structural: custom Azure OpenAI retrieval pipelines that bypass Azure AI Search index content under a broadly privileged service account with no query-time entitlement check unless the developer builds one. Azure AI Search's native document-level ACL trimming via Entra-backed tokens—available since May 2025—addresses this for deployments using that path, but Microsoft's own documentation confirms that if the permitted-groups field is not mapped, document-level access is disabled. Cioffi's fix was a query-path filter that checks the requesting user's SharePoint permissions before any content reaches the model's context window.
Our Take
This is the AI deployment risk that answer-quality evaluations were never designed to catch, which is exactly why it survives them. The evaluation asks whether the agent answers correctly. It does not ask whose permissions the retrieval step used. Every enterprise running a custom RAG pipeline against SharePoint—or any permission-tiered knowledge base—is exposed to the same failure until someone explicitly builds the entitlement check into the retrieval path. The CrowdStrike acquisition of SGNL and Palo Alto Networks' absorption of CyberArk address the credential lifecycle layer, but neither closes the retrieval permission boundary. Marketing teams that have deployed AI assistants against internal content libraries, CRM data, or campaign documentation stored in SharePoint face the same structural gap Cioffi found.
What Changed
Retrieval-time entitlement filtering now exists as a deployable control at the query-path layer—distinct from identity governance platforms, which manage credential lifecycle but cannot intercept the moment a correctly credentialed service account retrieves content on behalf of a user with fewer permissions than the indexer holds.
Marketing Impact
Marketing operations and martech teams running AI assistants against internal knowledge bases—brand guidelines, campaign briefs, pricing documents, customer data stored in SharePoint—are directly exposed. A low-privilege contractor or agency user querying an internal assistant could retrieve content they have no authorised access to.
Competitive Implication
Enterprise AI platform vendors with native retrieval-time ACL enforcement built into their indexing paths gain procurement advantage as this failure class becomes documented. Teams running custom RAG pipelines—common among sophisticated in-house build teams—carry latent liability that off-the-shelf, compliant-path deployments do not.
Strategic Outlook
Expect retrieval-time entitlement verification to become a standard line item in enterprise AI deployment audits through Q4 2026. Security vendors will productise the two-account test Cioffi ran manually. Organisations that cannot demonstrate query-path entitlement enforcement will face friction in procurement, legal review, and data governance audits.
The Exploit
Action Item
Marketing operations leaders with live RAG deployments against SharePoint should run Cioffi's two-account test this week: surface a question from a low-privilege account that a high-privilege account has already posed and compare outputs against what the low-privilege account can access directly in SharePoint.