feat(prism): similarity precheck API (3/coldkey/UTC day) - #89
Merged
Conversation
Let miners dry-run the intake copy gate before burning a 1-max slot or Lium pod. Cap at 3 checks per coldkey per UTC day so hotkey rotation cannot spam.
📝 WalkthroughWalkthroughAdds ChangesPRISM precheck
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant PrecheckAPI
participant PrismStore
participant PrecheckPipeline
Client->>PrecheckAPI: POST /v1/submissions/precheck
PrecheckAPI->>PrismStore: Consume daily quota
PrismStore-->>PrecheckAPI: Count or exhausted status
PrecheckAPI->>PrecheckPipeline: Evaluate ephemeral candidate
PrecheckPipeline-->>PrecheckAPI: PrecheckResult
PrecheckAPI-->>Client: JSON verdict and quota details
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/prism-challenge/src/api.rs`:
- Around line 287-289: Update the endpoint flow around evaluate_copy_precheck to
propagate corpus-loading errors instead of converting them to an empty list:
load recent entries with normal error handling, return an HTTP 500 on failure,
and only consume quota and evaluate the precheck after the corpus loads
successfully.
In `@crates/prism-pipeline/src/precheck.rs`:
- Around line 80-86: Update quota_identity to treat an all-zero coldkey as
absent before selecting PrecheckIdentityKind::Coldkey, falling back to the
trimmed, lowercased hotkey with HotkeyFallback. Add coverage for zero-owner
quota_identity results and API quota behavior ensuring distinct hotkeys do not
share the zero-owner quota key.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 3bcdc4f3-5f0d-4446-934e-ec2b57077902
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (12)
crates/db/migrations/0016_prism_precheck_quota.sqlcrates/db/src/prism_store.rscrates/prism-challenge/src/agentic.rscrates/prism-challenge/src/api.rscrates/prism-pipeline/Cargo.tomlcrates/prism-pipeline/src/lib.rscrates/prism-pipeline/src/precheck.rscrates/prism-store/src/dbprism.rscrates/prism-store/src/store.rsdocs/PRISM.mddocs/external-miner/prism.mddocs/external-miner/troubleshoot.md
Comment on lines
+287
to
+289
| let recent = st.store.list(None, None, 64).await.unwrap_or_default(); | ||
| let result = evaluate_copy_precheck(&candidate, &recent, quota); | ||
| Json(precheck_json(&result)).into_response() |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not return clean when corpus loading fails.
Line 287 converts a store failure into an empty corpus. The endpoint then returns 200 with similar: false after it consumes quota. This is not the same copy gate as intake.
Load the corpus with normal error handling before quota consumption. Return a 500 when loading fails.
Proposed fix
- let used = match st
+ let recent = match st.store.list(None, None, 64).await {
+ Ok(rows) => rows,
+ Err(e) => return json_err(StatusCode::INTERNAL_SERVER_ERROR, "store", &e.to_string()),
+ };
+ let used = match st
.store
.precheck_quota_try_consume(&identity, &day, PRECHECK_DAILY_LIMIT)
.await
@@
- let recent = st.store.list(None, None, 64).await.unwrap_or_default();
let result = evaluate_copy_precheck(&candidate, &recent, quota);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| let recent = st.store.list(None, None, 64).await.unwrap_or_default(); | |
| let result = evaluate_copy_precheck(&candidate, &recent, quota); | |
| Json(precheck_json(&result)).into_response() | |
| let recent = match st.store.list(None, None, 64).await { | |
| Ok(rows) => rows, | |
| Err(e) => return json_err(StatusCode::INTERNAL_SERVER_ERROR, "store", &e.to_string()), | |
| }; | |
| let result = evaluate_copy_precheck(&candidate, &recent, quota); | |
| Json(precheck_json(&result)).into_response() |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/prism-challenge/src/api.rs` around lines 287 - 289, Update the
endpoint flow around evaluate_copy_precheck to propagate corpus-loading errors
instead of converting them to an empty list: load recent entries with normal
error handling, return an HTTP 500 on failure, and only consume quota and
evaluate the precheck after the corpus loads successfully.
Comment on lines
+80
to
+86
| pub fn quota_identity(hotkey: &str, coldkey: Option<&str>) -> (String, PrecheckIdentityKind) { | ||
| match coldkey.map(str::trim).filter(|s| !s.is_empty()) { | ||
| Some(ck) => (ck.to_ascii_lowercase(), PrecheckIdentityKind::Coldkey), | ||
| None => ( | ||
| hotkey.trim().to_ascii_lowercase(), | ||
| PrecheckIdentityKind::HotkeyFallback, | ||
| ), |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Treat an all-zero Owner as unknown.
Line 81 accepts a zero coldkey as Coldkey. The documented unknown-owner case must use HotkeyFallback. Otherwise, all hotkeys with a zero Owner share one quota key, so one miner can exhaust another miner's precheck budget.
Filter the all-zero value before the coldkey branch. Add tests for both quota_identity and the API quota behavior.
Proposed fix
pub fn quota_identity(hotkey: &str, coldkey: Option<&str>) -> (String, PrecheckIdentityKind) {
- match coldkey.map(str::trim).filter(|s| !s.is_empty()) {
+ match coldkey
+ .map(str::trim)
+ .filter(|s| !s.is_empty() && !s.bytes().all(|b| b == b'0'))
+ {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| pub fn quota_identity(hotkey: &str, coldkey: Option<&str>) -> (String, PrecheckIdentityKind) { | |
| match coldkey.map(str::trim).filter(|s| !s.is_empty()) { | |
| Some(ck) => (ck.to_ascii_lowercase(), PrecheckIdentityKind::Coldkey), | |
| None => ( | |
| hotkey.trim().to_ascii_lowercase(), | |
| PrecheckIdentityKind::HotkeyFallback, | |
| ), | |
| pub fn quota_identity(hotkey: &str, coldkey: Option<&str>) -> (String, PrecheckIdentityKind) { | |
| match coldkey | |
| .map(str::trim) | |
| .filter(|s| !s.is_empty() && !s.bytes().all(|b| b == b'0')) | |
| { | |
| Some(ck) => (ck.to_ascii_lowercase(), PrecheckIdentityKind::Coldkey), | |
| None => ( | |
| hotkey.trim().to_ascii_lowercase(), | |
| PrecheckIdentityKind::HotkeyFallback, | |
| ), |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/prism-pipeline/src/precheck.rs` around lines 80 - 86, Update
quota_identity to treat an all-zero coldkey as absent before selecting
PrecheckIdentityKind::Coldkey, falling back to the trimmed, lowercased hotkey
with HotkeyFallback. Add coverage for zero-owner quota_identity results and API
quota behavior ensuring distinct hotkeys do not share the zero-owner quota key.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
POST /v1/submissions/precheckso miners can dry-run the intake copy gate (same AST/byte logic + same-hotkey/coldkey exclusion) without queuing a submission or renting a Lium pod.prism_precheck_quotawith 3 checks per coldkey per UTC day (hotkey fallback when Owner unknown); 4th call →429/precheck_quota_exceeded.docs/PRISM.md+docs/external-miner/; Design deferred (already has runtime copy-gate + daily manual quota).Test plan
cargo test -p prism-pipeline -p prism-challenge --lib(precheck unit + API tests)cargo clippy -p prism-pipeline -p prism-store -p prism-challenge --all-targets -- -D warningscargo run -p xtask -- loc-cap/external-docs-checkprism-challenge(migration0016_prism_precheck_quota)curl …/v1/submissions/precheckagainst staging/prod gatewaySummary by CodeRabbit
New Features
Documentation