Multi-level calls check() on every write, adding a remote read+write per compilation

[garysassano]

Summary

The multi-level backend calls check() on every cache write, for each level. For remote levels check() performs a remote read and a remote write of .sccache_check, so each cached compilation adds two extra remote requests per remote level before the real write is even queued. The result is not memoized.

Code

v0.17.0 (c037e11), write_remaining_levels_async — src/cache/multilevel.rs L599:

for (idx, level) in self.levels.iter().enumerate().skip(start_idx) {
    // Check if level is read-only before spawning task
    if matches!(level.check().await, Ok(CacheMode::ReadOnly)) {
        debug!("Level {} is read-only, skipping write", idx);
        continue;
    }
    // ... tokio::spawn(write)

RemoteStorage::check() reads .sccache_check and then writes it — src/cache/cache.rs L253-L295.

The same per-write check() pattern also appears at L777, L830, L860 and L882 of multilevel.rs.

By contrast the single-backend path resolves the cache mode once at server startup — src/server.rs L472 — which is why this is specific to the multi-level backend.

Related

#2070 and #2132 report .sccache_check causing rate limiting, and #2593 covers the read-only fallback when that check is rate limited. Those predate the multi-level backend; this issue is that the same check now also runs on the write path, once per compilation per remote level, rather than once per server.

Suggested fix

Resolve each level's CacheMode once (at construction, or behind a OnceCell) and reuse it for the lifetime of the storage, as the single-backend path already does.