Skip to content

fix(prism): fair review for generic LM tropes - #91

Merged
echobt merged 1 commit into
mainfrom
fix/prism-agentic-tops-only
Aug 8, 2026
Merged

fix(prism): fair review for generic LM tropes#91
echobt merged 1 commit into
mainfrom
fix/prism-agentic-tops-only

Conversation

@echobt

@echobt echobt commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Prod rejects for 6d0824f9… / b19998da… were cheap LLM similarity-v2 false positives (RMSNorm/RoPE/SwiGLU evidence at score 0.7), not AST copy-gate or agentic. AST vs peers/baseline is ~6100–7400 bps (under AST_SUSPICIOUS_BPS=8500).
  • similarity-v3 + parser coercion: generic modern-LM components are never plagiarism evidence; pre-pod hard-rejects Copied only (Suspicious advisory).
  • Copy / LLM similarity / agentic corpus = champions only (Score>0 tops + ex-tops) + baseline.
  • Public site gallery/leaderboard hide non-champions; operator challenge API still lists all.
  • Prod ops: reopened prism gating to open for both miner hotkeys (no scores invented).

Test plan

  • cargo test -p prism-review -p prism-challenge -p site-data -p site-api -p challenge-agentic -p prism-store -p prism-pipeline
  • CI green
  • Deploy prism-challenge + gateway (site filter); confirm precheck still works
  • Miners can submit again under same hotkeys

Summary by CodeRabbit

  • New Features

    • Public Prism galleries and leaderboards now show only current and historical champion submissions with positive scores.
    • Similarity reviews use champion architectures and baseline references while excluding standard language-model components as plagiarism evidence.
  • Bug Fixes

    • Suspicious similarity findings are now advisory instead of automatically producing a zero score.
    • Clear copying and agentic cheating or suspicious behavior continue to trigger hard-zero scoring.
    • Generic architecture terminology is no longer treated as sufficient evidence of copying.
  • Documentation

    • Updated Prism scoring, review, corpus, and public API guidance to reflect these changes.

Stop Score(0) on RMSNorm/RoPE/SwiGLU false positives; champions-only
copy corpus; public gallery shows tops/ex-tops only.
@echobt
echobt merged commit 697a7d2 into main Aug 8, 2026
0 of 2 checks passed
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bc67e59b-6ce6-40dd-aa83-8a63c7fd2cfc

📥 Commits

Reviewing files that changed from the base of the PR and between b0ef8b0 and 8b37134.

📒 Files selected for processing (21)
  • crates/challenge-agentic/src/prompts.rs
  • crates/db/src/prism_store.rs
  • crates/prism-challenge/src/api.rs
  • crates/prism-challenge/src/orchestrator.rs
  • crates/prism-challenge/src/score.rs
  • crates/prism-challenge/tests/copy_gate.rs
  • crates/prism-pipeline/src/precheck.rs
  • crates/prism-review/prompts/similarity_v3.md
  • crates/prism-review/src/generic_arch.rs
  • crates/prism-review/src/lib.rs
  • crates/prism-review/src/llm.rs
  • crates/prism-review/src/prompts.rs
  • crates/prism-store/src/dbprism.rs
  • crates/prism-store/src/store.rs
  • crates/site-api/src/handlers.rs
  • crates/site-data/src/map.rs
  • docs/COMPLETENESS.md
  • docs/PRISM.md
  • docs/PRISM_RECIPE.md
  • docs/SITE_API.md
  • docs/external-miner/prism.md

📝 Walkthrough

Walkthrough

PRISM now limits copy and review corpora to positive-score champion submissions, updates similarity review to v3, ignores generic language-model components as plagiarism evidence, makes suspicious similarity advisory, and exposes champions only in public listings.

Changes

PRISM review and champion filtering

Layer / File(s) Summary
Champion corpus retrieval and wiring
crates/prism-store/..., crates/db/..., crates/prism-challenge/..., crates/prism-pipeline/...
The store adds champion listing support. Copy, agentic, and similarity flows use champion submissions instead of general or terminated submissions.
Similarity prompt and generic-architecture handling
crates/prism-review/prompts/similarity_v3.md, crates/prism-review/src/..., docs/COMPLETENESS.md, docs/PRISM_RECIPE.md
Similarity v3 defines architecture-only judgments. Generic language-model evidence is coerced to Original under the defined confidence thresholds.
Scoring and anti-cheat verdict handling
crates/challenge-agentic/..., crates/prism-challenge/..., docs/PRISM.md, docs/external-miner/prism.md
Only Copied similarity results hard-gate scoring. Agentic Cheat and Suspicious results remain hard-zero outcomes.
Public champion filtering
crates/site-data/..., crates/site-api/..., docs/SITE_API.md
Public Prism submissions and BPB leaderboard entries now include positive-score champion submissions only.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Candidate as Candidate submission
  participant PrismStore
  participant SimilarityReview as Similarity review
  participant Scoring as Scoring gate
  Candidate->>PrismStore: Load positive-score champion corpus
  PrismStore-->>SimilarityReview: Return champion submissions
  SimilarityReview-->>Scoring: Return Copied, Suspicious, or Original
  Scoring-->>Candidate: Apply hard gate or advisory result
Loading

Possibly related PRs

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/prism-agentic-tops-only

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant