Skip to content

test(encryption): ignore the wall-clock key-timing test on shared runners (LAB-6714) - #95

Merged
27Bslash6 merged 1 commit into
mainfrom
lab-6714-ignore-key-timing-test
Sep 30, 2026
Merged

27Bslash6 merged 1 commit into
mainfrom
lab-6714-ignore-key-timing-test

Conversation

@27Bslash6

@27Bslash6 27Bslash6 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

This PR marks security_tests::test_timing_independent_of_key_pattern in tests/encryption_tests.rs with #[ignore], so it no longer runs by default in cargo test or in CI. The test body, its diff < 2.0 threshold, and all code under src/ are unchanged. No public APIs are affected.

Changes

  • Adds #[ignore = "wall-clock timing ratio on shared CI runners cannot measure constant-time behaviour"] to the test. The reason appears in the test runner output, so the exclusion is visible rather than silent.
  • Adds a comment above the test covering:
    • Wall-clock ratios on shared runners are noise, not constant-time measurements.
    • On native targets the timed operation is ring's AES-256-GCM, so the test cannot detect a leak in cachekit-core's own code.
    • The threshold was already raised 20% → 150% → 200%, and macos-latest still hit 209%. A link to the failing run is included.
    • Raising the threshold again would let a real 2x leak pass.
    • The exact command to run the test on demand:
      cargo test --all-features --test encryption_tests -- --ignored test_timing_independent_of_key_pattern

Impact

  • CI no longer fails intermittently on this timing check, particularly on macOS runners.
  • The test can still be run manually with --ignored under controlled conditions, with the original assertion strictness.
  • Other tests in security_tests are unaffected. That includes the preceding timing test, which still runs by default.

…ners (LAB-6714)

A wall-clock ratio on a shared CI runner cannot measure constant-time
behaviour, and on native targets the timed operation is ring's
AES-256-GCM, not this crate's code. The threshold was already raised
20% -> 150% -> 200% and macos-latest still hit 209%. Ignore it by
default with the on-demand command in a comment.
@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: cachekit-io/cachekit-core/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 8c577117-06bb-40fc-8d83-690be265d5e8

📥 Commits

Reviewing files that changed from the base of the PR and between 757c9d0 and 7c84798.

📒 Files selected for processing (1)
  • tests/encryption_tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

The timing test is now ignored by default. Comments explain the limits of its measurements on shared runners and show how to run it on demand. The test logic and timing assertions are unchanged.

Changes

Timing test execution

Layer / File(s) Summary
Document and opt-in the timing test
tests/encryption_tests.rs
The test is ignored by default. Comments describe the shared-runner measurement limits, the measured ring AES-256-GCM operation, and the command to run the test on demand. The measurement logic and assertions are unchanged.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 7c847

The timing test remains available on demand while routine runs skip unreliable wall-clock measurements. No substantive issue prevents merging.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: ignoring the wall-clock encryption timing test on shared runners. It includes the relevant test area and issue reference.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kodus-27b

kodus-27b Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

@27Bslash6
27Bslash6 merged commit a71fcc4 into main Sep 30, 2026
34 checks passed
@27Bslash6
27Bslash6 deleted the lab-6714-ignore-key-timing-test branch September 30, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant