test(byte_storage): make the compression_bomb fuzz target and Kani bound proofs able to fail (LAB-5508)

[27Bslash6]

The decompression bound in StorageEnvelope::extract was sound, but its checks could be deleted without failing anything: the compression_bomb fuzz target could not build an input only a size cap rejects, the Kani size/ratio harnesses compared a predicate to itself, and no merge-time test covered the compressed-length cap. This PR makes each of the three checks fail the fuzz target, a Kani proof and a cargo test when it is deleted. It does not change behaviour.

What changed

• src/byte_storage.rs. The three checks move, unchanged and in the same order, into a private check_decompression_bound(compressed_len, original_size). extract calls it first. The constants and error variants are untouched, and every existing test passes without modification.

• Kani. verify_decompression_bound_size_caps and verify_decompression_bound_ratio call check_decompression_bound over every (usize, u32) pair and compare it with limits written as literals. So an inverted comparison or a changed constant fails a proof. The four tautological harnesses (verify_decompression_bomb_protection, verify_input_size_limits, verify_compressed_size_limits, verify_compression_ratio_calculation_safety) are deleted.

• Unit test. test_extract_rejects_oversized_compressed_data calls extract with compressed_data.len() == 512 MiB + 1 and original_size == 1, and requires InputTooLarge.

• fuzz/fuzz_targets/compression_bomb.rs is rewritten as an oracle. Every input gets one expected result, and extract and retrieve must return exactly that. There are no guard-gated asserts and no catch-all Err arm. Classes:

  • CompressedOverCap: length in (512 MiB, 512 MiB + 256]. Only the compressed-length cap rejects it.
  • OriginalOverCap: length ≥ 536,871, original_size in (512 MiB, 1000 × len]. Only the original_size cap rejects it.
  • RatioOver: both sizes under their caps, ratio over 1000:1.
  • CompressedAtCap: length exactly 512 MiB. extract gets past the bound and fails at decompression.
  • Valid: a stream from StorageEnvelope::new, with the declared size replaced and/or the checksum altered. This reaches ChecksumMismatch, SizeValidationFailed and DecompressionFailed.
  • Raw: arbitrary bytes, declared size and checksum. The expected result comes from lz4_flex::decompress plus cachekit_core::checksum. lz4_flex is added to the fuzz crate only, at the version cachekit-core already resolves (cargo tree -i lz4_flex shows one copy).

  The size-class payloads are zeroed, so they stay lazily mapped. For the two 512 MiB classes, retrieve gets zeroed bytes just over its length cap rather than a serialized envelope. It must return InputTooLarge. Without the length check those bytes decode to DeserializationFailed. So the target runs at about 18k to 25k exec/s, with peak RSS around 400 MB, under libFuzzer's default limit.

• .github/workflows/security.yml (deep-fuzz only). Each target's fuzz/corpus/<target> is uploaded after the run as an artifact named fuzz-corpus-<target>, kept for 90 days, under always(). Before the run, the job restores the newest unexpired artifact of that name from this repository on the same ref, using actions/download-artifact with run-id. If there is none, it emits a ::warning:: and starts empty. The job gains actions: read for that lookup. An artifact is used because Actions cache entries are evicted under size pressure and after 7 days unused, which a weekly run cannot outlast.

• SECURITY.md. The "Test coverage" paragraph is rewritten to match the above. It names tests/byte_storage_tests.rs and scopes its Kani statements to the two proofs.

Evidence

Every command ran at 0b7ba9d. Each fuzz run starts from a fresh empty corpus directory (cargo fuzz run compression_bomb <empty-dir> -- -max_total_time=120).

Unmutated head:

cargo test --all-features   → 237 passed, 0 failed (main: 236; +1 new test)
cargo kani --all-features   → Complete - 9 successfully verified harnesses, 0 failures, 9 total.
cargo +nightly fuzz run compression_bomb -- -max_total_time=120
                            → Done 3066592 runs in 121 second(s); peak rss 410Mb; exit 0

Mutation matrix. One check deleted per row. Line numbers are in check_decompression_bound on this branch.

Deleted	cargo fuzz run (120 s, empty corpus)	cargo kani --all-features	cargo test --all-features
compressed-length cap, :139-141	exit 1. extract: expected Err(InputTooLarge), got Err(DecompressionFailed) (compressed_len=536870953 original_size=31242)	exit 1. verify_decompression_bound_size_caps FAILED: assertion failed: over_cap == matches!(result, Err(ByteStorageError::InputTooLarge)). 8 of 9 verified	exit 101. test_extract_rejects_oversized_compressed_data FAILED (109 passed, 1 failed)
original_size cap, :143-145	exit 1. extract: expected Err(InputTooLarge), got Err(DecompressionFailed) (compressed_len=587243 original_size=536873669)	exit 1. verify_decompression_bound_size_caps FAILED (same check). 8 of 9 verified	exit 101. test_decompression_bomb_integer_boundary, test_decompression_u32_max_original_size FAILED
ratio comparison, :162-164	exit 1. extract: expected Err(DecompressionBomb), got Err(DecompressionFailed) (compressed_len=65535 original_size=65572129)	exit 1. verify_decompression_bound_ratio FAILED: assertion failed: matches!(result, Err(ByteStorageError::DecompressionBomb)). 8 of 9 verified	exit 101. test_compression_ratio_bomb_protection, test_decompression_bomb_extreme_ratio, test_decompression_just_over_threshold FAILED

Reach checks (same fuzz command, not part of the matrix):

• Delete the checksum check in extract: exit 1, expected Err(ChecksumMismatch), got Ok(..).
• Delete the final size check: exit 1, expected Err(SizeValidationFailed), got Ok(..).
• Delete retrieve's envelope-length check: exit 1, retrieve: expected Err(InputTooLarge), got Err(DeserializationFailed("invalid type: integer 0, expected struct StorageEnvelope")).

Corpus persistence: two workflow_dispatch runs with run_deep_fuzz=true and fuzz_seconds=60; see the latest comment below.

Summary

This PR fixes the fuzz corpus artifact lookup in the security workflow and aligns documentation with how the compression_bomb fuzz target and its corpus handling actually behave. No public APIs are modified.

Changes

CI: fuzz corpus artifact lookup (.github/workflows/security.yml)

The step that finds the previous fuzz corpus artifact for the current branch has been restructured:

• Pagination: gh api now uses --paginate, so every page of artifacts is searched. Previously only the first 100 results were checked, and newer artifacts from other branches could push this branch's corpus out of view.
• Failure handling: Listing the artifacts and selecting the run ID are now separate commands. If the API listing fails, the step fails under set -e. Previously a failure was silently treated as "no artifact found."
• Selection logic is unchanged: it still picks the newest non-expired artifact whose head branch and repository match the current ref. When none exists, it still warns and falls back to an empty corpus.

Documentation (SECURITY.md)

The "Test coverage" section is revised:

• The fuzz target is described as computing one expected result for each call to extract and retrieve, rather than one result per input that both must share.
• The corpus lifecycle is stated explicitly. Each scheduled or on-demand deep run uploads its corpus as an artifact kept for 90 days. The next run on the same branch starts from the newest one, or warns and starts empty if none exists.
• The remaining changes are line reflow only. The Kani proof descriptions are unchanged in content.

Fuzz target docs (fuzz/fuzz_targets/compression_bomb.rs)

The module doc comment is updated to the same per-call wording. The target's code is unchanged.

Public API impact

None. Changes are limited to CI configuration, documentation, and a doc comment.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: cachekit-io/cachekit-core/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 734cf963-6669-4404-ac13-3d3d82b228e1

📥 Commits

Reviewing files that changed from the base of the PR and between ac67c57 and cde10a9.

📒 Files selected for processing (4)

• .github/workflows/security.yml
• SECURITY.md
• fuzz/fuzz_targets/compression_bomb.rs
• src/byte_storage.rs

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: cachekit-io/cachekit-core/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 03610f60-7e8a-4faa-98c7-edd925a6d739

📥 Commits

Reviewing files that changed from the base of the PR and between f29965d and ac67c57.

📒 Files selected for processing (5)

• .github/workflows/security.yml
• SECURITY.md
• fuzz/Cargo.toml
• fuzz/fuzz_targets/compression_bomb.rs
• src/byte_storage.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

---

Walkthrough

The change centralises decompression-bound checks in the extraction path and expands fuzz cases and expected-result checks. The deep-fuzz workflow now restores and saves target-specific corpora, sets memory limits, and passes them to libFuzzer. The security documentation describes test and proof coverage.

Changes

Decompression Bound Checks and Verification

Layer / File(s)	Summary
Production decompression-bound checks src/byte_storage.rs	extract calls check_decompression_bound before decompression. A new test checks rejection of compressed data above the size cap. Kani proofs exercise the production helper with the size and ratio limits.
Fuzz cases and expected outcomes fuzz/Cargo.toml, fuzz/fuzz_targets/compression_bomb.rs	The fuzz target adds cases for malformed inputs, altered metadata, and size and ratio boundaries. It models expected results and checks both extract and retrieve.
Deep-fuzz workflow and coverage documentation .github/workflows/security.yml, SECURITY.md	The workflow restores and saves target-specific corpora, sets memory limits, and passes the limit to libFuzzer. The documentation describes test coverage and the scope of two Kani proofs.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to ac67c

The change strengthens decompression-bound validation without an identified runtime behavior regression. No actionable merge-blocking risk is established; merge after normal checks pass.

Security Architecture Review

Security architecture risk: 🔵 Low · up to ac67c

Existing decompression safeguards and output handling are preserved. The workflow changes affect security testing rather than production deployment. External caller and deployment exposure remain unverified, so the assessment is low rather than minimal.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The supported resource scope is the embedding caller's process: envelope decoding and decompression consume its memory and CPU. The unchanged 512 MiB size caps and 1000:1 ratio limit constrain individual operations, not aggregate concurrent demand. Tenant, service, and environment exposure are not established.

Security Findings and Attack Paths

• inferred — If an embedding caller accepts attacker-supplied envelopes, those bytes reach decoding and decompression through the C wrapper. The inspected base/head path preserves its controls; no PR-introduced bypass is established. External attacker reachability remains unresolved rather than a verified finding.

Trust Boundaries and Controls

• observed — Validation remains in the storage layer, while the C wrapper maps errors and copies successful output into caller-owned memory. Centralization does not transfer validation responsibility, introduce caller authentication, or change allocation ownership.

Resilience and Maintainability Implications

• observed — Extraction errors return without copying output. BufferTooSmall reports the required length without copying; success copies the validated bytes and updates length. Intermediate Rust allocations remain local to each call, preserving existing cleanup and repetition behavior. Concurrent reuse of caller buffers remains the caller's responsibility.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 2 files. (3 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: strengthening the compression_bomb fuzz target and Kani bound proofs so that they can detect removed checks. The issue reference is relevant.

Full details: Docstring Coverage

Explanation

Docstring coverage is 78.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 2 files. (3 skipped: 3 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kodus-27b]

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration

Interacting with Kody

• Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

• Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

• Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
• Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
• Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
• Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
• API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
• Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
• Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
• Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
• Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
• Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).

Current Kody Configuration

Review Options

The following review options are enabled or disabled:

Options	Enabled
Bug	✅
Performance	✅
Security	✅
Business Logic	✅

Access your configuration settings here.

Kody Code Review — 1 suggested fix.
Paste the prompt below to your agent and all review fixed at once!

🛠️ Open Agent Prompt

A code review identified the following issues in this pull request.
Each section describes what was found and includes a reference implementation where available.

Files involved:
- fuzz/fuzz_targets/compression_bomb.rs:191

---

### [1/1] fuzz/fuzz_targets/compression_bomb.rs:191
Issue identified during code review:
Throughput bottleneck in the compression_bomb fuzz_target: the CompressedOverCap and CompressedAtCap variants build a filled buffer of 512 MiB or more on every execution and serialize it with rmp_serde::to_vec, and CompressedAtCap also runs lz4_flex::decompress over it twice (expected_extract and extract). Arbitrary picks among the 6 variants roughly evenly, so about a third of mutated inputs each cost hundreds of milliseconds and about 1 GiB of memory traffic, which caps the 1-hour deep run at few executions and starves the Raw and Valid cases that explore malformed LZ4. Fix: these boundary cases depend only on a u8/u16 and are deterministic, so run each once per process behind a static AtomicBool flag (or move them to a unit test) and keep the fuzz loop on the cheap variants.
Reference implementation (from code review):

// fuzz/fuzz_targets/compression_bomb.rs:191
Case::CompressedOverCap { extra, original_size, fill } => {
            // Deterministic boundary check: run once per process.
            static DONE: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
            if DONE.swap(true, std::sync::atomic::Ordering::Relaxed) { return; }
            let len = MAX_COMPRESSED_SIZE + 1 + extra as usize;
            let envelope = envelope(vec![fill; len], original_size as u32);
            assert_outcome(&storage, &envelope, Err(ByteStorageError::InputTooLarge));
        }

---

Review each issue in context, use the reference implementations as guidance, and apply fixes that are consistent with the surrounding codebase.

[27Bslash6]

The deep-fuzz corpus now carries over between runs. I dispatched two workflow_dispatch runs at ac67c57 with run_deep_fuzz=true and fuzz_seconds=60. Both concluded with 17 jobs successful and 3 skipped, and Kani Formal Verification passed in both.

Run 1 (36668548107), Deep Fuzzing (compression_bomb). It started cold, as expected for the first run, and saved its corpus:

Cache not found for input keys: fuzz-corpus-compression_bomb-36668548107-1, fuzz-corpus-compression_bomb-
INFO: A corpus is not provided, starting from an empty corpus
Done 1477 runs in 61 second(s)
Cache saved with key: fuzz-corpus-compression_bomb-36668548107-1

Run 2 (36668784168), Deep Fuzzing (compression_bomb). It restored run 1's corpus and started from it:

Cache hit for restore-key: fuzz-corpus-compression_bomb-36668548107-1
INFO:       37 files found in /home/runner/work/cachekit-core/cachekit-core/fuzz/corpus/compression_bomb
INFO: seed corpus: files: 37 min: 7b max: 8b total: 283b rss: 40Mb
Done 424 runs in 61 second(s)
Cache saved with key: fuzz-corpus-compression_bomb-36668784168-1

[kodus-27b]

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration

Interacting with Kody

• Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

• Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

• Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
• Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
• Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
• Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
• API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
• Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
• Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
• Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
• Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
• Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).

Current Kody Configuration

Review Options

The following review options are enabled or disabled:

Options	Enabled
Bug	✅
Performance	✅
Security	✅
Business Logic	✅

Access your configuration settings here.

[27Bslash6]

At 0b7ba9d the corpus is carried between runs as a 90-day artifact instead of an Actions cache entry. That replaces the cache-based evidence in my earlier comment. I dispatched two workflow_dispatch runs with run_deep_fuzz=true and fuzz_seconds=60. Both concluded with 17 jobs successful and 3 skipped, and Kani Formal Verification passed in both.

Run 1 (36670689677), Deep Fuzzing (compression_bomb). No artifact existed yet, so it warned, started cold, and uploaded its corpus:

##[warning]No fuzz-corpus-compression_bomb artifact from agent/miss-huang/lab-5508-compression-bomb-coverage; starting from an empty corpus
INFO: A corpus is not provided, starting from an empty corpus
Done 1191482 runs in 61 second(s)
Artifact fuzz-corpus-compression_bomb has been successfully uploaded! Final size is 97136 bytes. Artifact ID is 11076939797

Run 2 (36670938699), Deep Fuzzing (compression_bomb). It restored run 1's artifact and started from it:

run-id: 36670689677
Artifact download completed successfully.
INFO:      432 files found in /home/runner/work/cachekit-core/cachekit-core/fuzz/corpus/compression_bomb
INFO: seed corpus: files: 432 min: 7b max: 3990b total: 125076b rss: 41Mb
Done 1053258 runs in 61 second(s)
Artifact fuzz-corpus-compression_bomb has been successfully uploaded! Final size is 172805 bytes. Artifact ID is 11077758137

The zero-filled size classes raised this target from 1,477 runs per 61 s to about 1.1 million. The slow-unit report that the earlier run 2 uploaded no longer appears.

[kodus-27b]

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration

Interacting with Kody

• Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

• Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

• Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
• Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
• Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
• Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
• API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
• Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
• Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
• Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
• Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
• Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).

Current Kody Configuration

Review Options

The following review options are enabled or disabled:

Options	Enabled
Bug	✅
Performance	✅
Security	✅
Business Logic	✅

Access your configuration settings here.

[27Bslash6]

At cde10a9, the corpus lookup lists artifacts and selects one in two separate steps. If the listing fails, the step now fails instead of reading as "no artifact" and starting cold. The listing also walks every page, so newer artifacts from other branches cannot push this branch's corpus off the first page.

Dispatch 36672450347 (run_deep_fuzz=true, fuzz_seconds=60) concluded with 17 jobs successful and 3 skipped. Deep Fuzzing (compression_bomb) restored the previous run's corpus:

run-id: 36670938699
Total of 1 artifact(s) downloaded
INFO: seed corpus: files: 739 min: 7b max: 3990b total: 264628b rss: 43Mb
Done 1263056 runs in 61 second(s)
Artifact fuzz-corpus-compression_bomb has been successfully uploaded! Final size is 230952 bytes. Artifact ID is 11078119583