When access is revoked, how fast does RAG forget?

Status: a reference design, not deployed as described. It is written from public platform documentation and describes no organisation’s systems, data or metrics; every example is generic. Every number is cited, derived from a formula on this page, or labelled as an assumption. Try it: the interactive walkthrough steps through a revocation from source to answer.

1. Two people, one question

Two people ask an enterprise assistant the same question. They should not get the same answer. One of them can read the restructuring plan; the other cannot. A retrieval-augmented assistant gets this right on day one if every chunk in its index carries the people and groups allowed to read it, and every query is filtered by the caller.

Day one is the easy day. After it, the source keeps changing. Someone removes a share. A manager takes a leaver out of a group. A site owner breaks inheritance on a folder. Each of those is a revocation, and the index hears about none of them on its own. On the reference platform the documentation says so directly: permission changes in the source appear in search results only after the permission metadata is synchronised into the index [1].

So I measure one number: time-to-revoke, the time from a revocation at the source to the moment the revoked person stops seeing that content in answers. The whole design exists to bound it.

An index is a copy of permissions. Every copy has an age.

It matters now because assistants are leaving pilots for the whole organisation. Fifty pilot users hide revocation lag. An assistant open to everyone turns every stale ACL into a question someone can ask. OWASP names unauthorised access and cross-context leakage from shared vector stores as a risk in its 2025 list for LLM applications [9].

2. Why the obvious answers fail

“We filter by the groups in the user’s token.” A token’s groups claim is a snapshot from when the token was issued. Default access tokens last an hour, and sessions under continuous access evaluation can hold tokens for up to 28 hours [7]. Continuous access evaluation acts on a short list of critical events (an account disabled or deleted, a password reset, MFA enabled, tokens revoked by an administrator, high user risk). A group membership change is not on it, and the documentation says such changes can take up to a day to take effect [7]. The claim also caps out: a JWT carries at most 200 groups before it is replaced by an overage indicator and the app has to ask the directory [6]. Filtering on the token is filtering on yesterday’s org chart.

“The indexer picks up permission changes.” Partly. The reference platform’s SharePoint connector picks up a change on an item with its own unique permissions on the next successful indexer run. A change on a parent (site, library, list or folder) that children inherit is not detected; you must call a resync or reset and run the indexer again, and until you do, the index serves stale ACLs [2]. Inheritance is how most organisations share, so the common case is the undetected one.

“Re-index nightly.” That bounds time-to-revoke at about a day plus the run time. For a leaver, that is exactly the window that matters.

“Check the source for every result.” The most correct answer and the most expensive. It adds a permission call per candidate chunk per query, and ties the assistant’s availability to every source’s API. I use it, but only for a short list of sensitive labels.

3. The design

Eleven components in four zones, plus the person asking. The sources and the directory are authoritative and outside my trust boundary. Everything from the sync service inward is mine and must assume it can be stale. Following zero-trust practice, nothing is trusted for where a request comes from; each query is authorised on its own [11].

4. A revocation, step by step

1. Two users, same question, different evidence. A and B ask about next year’s team structure. Each closure comes from the cache. A’s includes the planning group; B’s does not. A’s answer cites the plan; B’s cites the published org chart.
2. A share is revoked and the change reaches the index. The plan’s owner removes A’s direct share. The connector emits a permission event. The sync service writes a deny entry for (A, plan) in seconds, recomputes the plan’s ACL and rewrites every chunk with a new acl_version.
3. The answer changes. A asks again. Even if the index write has not landed, the overlay removes the plan’s chunks, and the cached answer misses because the plan’s epoch moved. A gets the org-chart answer.
4. A group member is removed. A leaves the planning group. The notification arrives, or the next poll finds it, and A’s closure is replaced. A’s token still lists the group. It no longer matters.
5. A sync breaks. No delta poll has succeeded within the staleness budget. Closures are marked stale and retrieval narrows to content everyone in scope may read, with a visible notice. An alert fires.
6. The canary measures it. A synthetic document is shared, revoked and probed until it disappears. That run’s time-to-revoke goes into the SLO.
7. Before go-live: the oversharing report. The scanner finds a library shared with everyone internally that holds personnel documents. It is excluded until its owner narrows access. Launch waits on high findings, not on all of them.

5. The subtle failure: the new enemy

Google’s Zanzibar paper describes the “new enemy” problem: a user is removed from a folder’s ACL, new documents are then added to the folder, and if the system applies the two changes out of order, the removed user sees the new documents [10]. A RAG index has the same hazard. A content update can re-index a chunk with an ACL read just before the revocation landed, and silently restore access.

The rule that prevents it: ACL and content updates for a document go through one ordered queue, and any ingest carrying an acl_version older than the stored one is rejected. Zanzibar solves the general case with consistency tokens; a per-document version is enough here.

6. Controls and failure modes

7. Decisions and trade-offs

Own the ACL sync; use the connector for content. The platform’s native ACL ingestion is attractive: query-time trimming with the user’s token, with nested Entra groups resolved through Graph [1][2]. But it is in preview, which the documentation says carries no SLA and is not recommended for production [1], and it misses inherited changes [2]. So I push principal IDs as plain security-filter fields, the generally available pattern [3], from my own sync. The cost is an inheritance walker I own. The gain is that every stage can be measured and bounded. When native ingestion reaches general availability with inherited-change detection, I would revisit this.

Server-side closure, never token claims. It costs a cache and a feed. It removes the 200-group overage problem [6] and decouples revocation from token lifetime [7].

A deny overlay instead of a synchronous re-index. Putting an index write on the critical path of every revocation is simple, but slow under load. The overlay makes the fast path one key write and lets the index catch up.

Fail closed. When the closure is stale beyond budget, users see less, with a notice. Serving on stale permissions is the failure the design exists to prevent.

Live re-check only for high labels. Checking every candidate against its source multiplies source calls by top-k. A short label list keeps cost proportional to risk.

8. Metrics and SLOs

Time-to-revoke. For a revocation at source time t_r, and the first time t_e after which no answer for the revoked principal cites the document: TTR = t_e − t_r. The canary measures it.

The budget (assumptions to tune, not benchmarks). Without the overlay, worst-case time-to-revoke for a group removal is the sum of the stage delays: TTR_max = P + I + T, where P is the delta poll interval, I the index write-to-visible allowance and T the closure cache TTL. With P = 5 min, I = 1 min and T = 5 min, TTR_max = 5 + 1 + 5 = 11 min. Against a 15-minute target that leaves 4 minutes for directory replication delay, which the documentation does not quantify [4][5]. With the overlay, an item-level revocation seen by the connector skips I and T: time-to-revoke is roughly connector detection plus one overlay write.

9. Rollout

1. Inventory and scan. Run the oversharing baseline [8], list unsupported principal types [2], set the exclusion list.
2. Shadow sync. Run the sync and the closure cache beside the existing index; nightly reconciliation only.
3. Canaries on. Publish time-to-revoke p95 for two weeks before any user sees an answer.
4. Pilot, fail-closed. Page on any stale-ACL leak.
5. Widen one source at a time, each with its own canary.
6. Re-scan on a cadence: snapshots quarterly, activity reports monthly, as the platform documentation recommends [8].

10. What it does not do

• It does not fix oversharing at the source. If a folder is shared with everyone, the assistant will answer everyone. The scanner finds it; owners fix it.
• It does not make revocation instant. It makes it bounded and measured.
• It does not recall answers already given. Someone who read a document before the revocation read it.
• It does not stop copying. A user can paste what they can read, or screenshot an answer.
• It does not handle embedding inversion or memorisation. It assumes source content never fine-tunes the model [9].
• It does not guess at principals the connector cannot express. That content is excluded.
• It is a reference design, not deployed as described. The budget figures are assumptions for the canary to replace with measurements.

Closing

Every permission-aware RAG system filters. The difference is whether anyone knows how old the permissions it filters on are. Resolve groups on the server, put a deny overlay in front of the index, keep ACL and content in order, fail closed when the copy is too old, and let a canary tell you the number. Then the answer to “how fast does RAG forget?” is a measured p95, not a hope.

Sources

1. Document-level access control in Azure AI Search, Microsoft Learn (updated 17 Sep 2026)
2. SharePoint ACL ingestion in Azure AI Search (preview), Microsoft Learn (updated 17 Sep 2026)
3. Security filters for trimming results in Azure AI Search, Microsoft Learn (updated 24 Aug 2026)
4. Use delta query to track changes in Microsoft Graph data, Microsoft Learn
5. Microsoft Graph change notifications overview, Microsoft Learn
6. Configure group claims and app roles in tokens, Microsoft Learn
7. Continuous access evaluation, Microsoft Learn (updated 8 Apr 2026)
8. Data access governance reports for SharePoint sites, Microsoft Learn (updated 30 Jul 2026)
9. LLM08:2025 Vector and Embedding Weaknesses, OWASP GenAI Security Project
10. Zanzibar: Google’s Consistent, Global Authorization System, Pang et al., USENIX ATC 2019
11. SP 800-207, Zero Trust Architecture, NIST, 2020