What scanning ~1M files against Microsoft Graph actually looks like (measured, not guessed)

Most posts about Graph throttling stop at “handle 429s and respect Retry-After.” That advice is correct and nowhere near enough. We recently ran a full SharePoint governance scan of a real ~900K-file tenant (a couple hundred sites, one drive holding over half a million files by itself) and instrumented every physical 429 at the wire. Here is what the throttling model actually does to a large scan — and why the obvious fix, “just add more app registrations,” hits a wall Microsoft built on purpose.

There are two throttles, and they behave nothing alike

When you scan SharePoint content through Graph you are living under two separate regimes:

1. Directory/identity throttling. Users, groups, and directory reads ride Microsoft Graph’s identity-service buckets — short windows, per-app, documented in Microsoft Graph service-specific throttling limits. These 429s are frequent, cheap, and bursty. In our scan the directory phases absorbed several hundred 429s and barely moved wall-clock time: back off a few seconds, continue.

2. SharePoint resource-unit (RU) throttling. Calls touching sites, drives, driveItems, and permissions are governed by a completely different system: SharePoint’s resource-unit quotas, documented in Avoid getting throttled or blocked in SharePoint Online. Every app gets an RU budget per minute and per 24 hours, scaled by the tenant’s license count (from 1,250 RU/min at the smallest tier to 6,250 RU/min at 50K+ licenses). This is the throttle that decides whether your scan takes hours or days.

If your telemetry doesn’t distinguish these two, your retry tuning is guesswork. Ours showed the split starkly in one run: ~600 directory 429s (noise) versus ~70 SPO RU backoffs (the actual constraint) — and they landed on different phases with different Retry-After behavior.

Where the RU budget actually goes

RU costs per operation are published in the same SharePoint doc, and they are wildly asymmetric:

• Delta query with a token: 1 RU. Deliberately discounted — Microsoft rewards incremental scanning.
• Enumeration / multi-item reads: 2 RU per request. Enumerating ~1M files at ~200 items/page costs roughly 10,000 RU total — under two minutes of a single app’s budget at the top tier. File enumeration is latency-bound, not quota-bound.
• Any permission operation: 5 RU. One permission read per item on ~1M items ≈ 5,000,000 RU. This is the wall. At mid-tier budgets a single app cannot even finish inside the 24-hour cap.

Practical consequence: your “1M-file scan” is really a cheap metadata scan stapled to a brutally expensive permissions scan. Optimize the permissions phase or nothing else matters.

What a throttled phase looks like from the inside

Measured on the permissions phase of our scan: an unthrottled batch round completed in ~24 seconds; the identical round under throttling took ~12 minutes 44 seconds — a ~32× slowdown — with 151 logged 429s and a strikingly uniform Retry-After of ~64 seconds. Two operational lessons:

1. Throttled requests still count against your quota. Aggressive retry is self-harm; the doc says so explicitly. Honor Retry-After exactly.
2. SDKs eat 429s silently. The Graph SDK’s built-in retry handler will hide throttling from your application logs entirely. Put a logging handler at the wire (a DelegatingHandler below the retry middleware) or you will believe your scan is “slow” rather than “throttled” — and tune the wrong thing.

Also measurable and worth knowing: the RU ledger resets meaningfully via the delta discount. Our second scan’s file phase — same tenant, delta tokens in hand — took 26 seconds where the first took over an hour. First scans are the hard problem; steady state is nearly free if you keep your tokens.

Why “just add app registrations” has a hard 3× ceiling

The tempting move: if the RU budget is per-app, register N apps and multiply the budget. Two things stop you.

The math stops you at 3. The per-app budgets are per-app per-tenant, but there is also a tenant-wide bucket shared by all applications — and at every license tier, the tenant-wide per-minute budget is exactly 3× the per-app budget. Three identities running flat-out consume the entire tenant’s SharePoint API budget, including what OneDrive sync, backup tools, and DLP scanners in that tenant need. Identity #4 adds nothing in the constrained window.

Microsoft stops you by policy. The throttling doc names this exact pattern under “Creating multiple AppIDs for the same application”: apps performing the same operations across multiple registrations “ended up exhausting the tenant’s resources,” and SharePoint “may completely block the application” — with the block notice landing in the customer’s Message Center. For a governance vendor, getting your customer an abuse notice is the end of the relationship.

There is a defensible middle path: a small number of identities partitioned by distinct workload (file inventory vs. permission analysis vs. directory/governance reads), each with least-privilege scopes, self-capped to a fraction of the tenant bucket so co-tenant applications are never starved. In our measured case, moving the 5-RU permissions workload onto the fleet takes a ~900K-item permissions backlog from ~3.7 days (measured, single app, 24-hour-cap-bound) to a projected ~1.3 days — and even that only works if the throttled workload’s identities are actually parallelized, which we learned the hard way when a role-partitioned fleet left the expensive phase on one identity and delivered no speedup at all. Fleet design is about which identity carries the RU-heavy phase, not how many identities exist.

The sanctioned alternatives

• Delta queries with tokens (1 RU, incremental) — mandatory, not optional, for steady state.
• Traffic decoration — the ISV|Company|App/Version User-Agent format. Microsoft states decorated traffic is prioritized. Check your client actually sends it; ours was configured for months and never transmitted until we verified at the wire.
• RateLimit-* headers — SharePoint emits IETF rate-limit headers when you’ve consumed ≥80% of a budget. Proactive slowdown beats reactive backoff.
• Microsoft Graph Data Connect — for very large tenants, the bulk path Microsoft itself recommends in the throttling doc: batch dataset extraction with zero API throttling, priced at $0.75 per 1,000 objects, with the SharePoint Files dataset discounted to $0.75 per 50,000 objects. Above a few million items, API scanning stops being the right tool for a first baseline.

Practitioner takeaways

1. Instrument physical 429s below the SDK retry layer, tagged by workload — directory noise and SPO RU pressure need different responses.
2. Budget in RUs before you scan: enumeration ≈ 2 RU/page, permissions = 5 RU/op. The permissions phase is your scan duration.
3. Never scale past 3 identities; the tenant-wide bucket is 3× per-app at every tier, and the dozen-app pattern is the documented abuse case.
4. If you do run multiple identities, self-cap total consumption below the tenant budget — you share it with the customer’s other tooling.
5. Protect delta tokens like state, because they are: they’re the difference between a 63-minute phase and a 26-second one.
6. Above ~5M items, price out Graph Data Connect before building a bigger scanner.

We build governance tooling for the agentic Microsoft 365 estate, and everything above is measured from our own collectors. Sources: SharePoint Online throttling guidance, Microsoft Graph throttling limits, MGDC pricing update.