Skip to content

Keep the compilation cache within CUDA_TILE_CACHE_SIZE - #114

Open
0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:cache-evict-respect-size-limit
Open

0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:cache-evict-respect-size-limit

Conversation

@0z5a

@0z5a 0z5a commented Nov 3, 2026

Copy link
Copy Markdown

Description

evict_lru deletes the oldest entries while their running total is at most total - limit. It keeps the entry that crosses that point, so the cache can stay over the limit: with entries of 300, 300 and 400 bytes and a 600-byte limit, 700 bytes remain; with two 600-byte entries and a 1000-byte limit, nothing is evicted. The existing test uses equal-size entries that land exactly on the boundary.

An entry is now evicted when the entries older than it do not free enough space. One side effect: a single entry larger than the whole limit is evicted right after it is stored.

test_evict_lru_respects_size_limit covers five cases; three fail before this change. test/test_cache.py passes (15 tests, CPU only).

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

evict_lru deleted the oldest entries only while their running total was
at most (total size - limit), so it kept the entry that crosses that
point and the cache could stay over the limit. For example, with entries
of 300, 300 and 400 bytes and a 600-byte limit, 700 bytes remained.
Delete an entry when the entries older than it do not free enough space.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant