fix(similarity): exclude same-coldkey prior art (Design + Prism) - #82
Conversation
Hotkey 1-max gating forces miners to iterate under new hotkeys of the same coldkey; treat those revisions as self, not cross-miner copies, in gate and review corpora. Persist Owner coldkey at intake from the cached metagraph.
📝 WalkthroughWalkthroughThe PR adds coldkey data to metagraphs, caches, harnesses, and submissions. It persists coldkeys and applies hotkey/coldkey miner identity checks when building copy-gate and review corpora. ChangesMetagraph coldkey loading
Coldkey persistence and capture
Same-miner corpus filtering
Fixtures and documentation
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
crates/chain-live/src/tests.rs (1)
658-659: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMock actual
Ownerkeys and assert resolved coldkeys.The mocked changes contain
Keysstorage keys, notOwnerstorage keys. Therefore,fetch_coldkeys_for_hotkeyscannot match either response, andcoldkeysremains zero-filled. Line 659 checks only the vector length.Return
storage_map_key_account_blake2("SubtensorModule", "Owner", ...)keys with distinct coldkeys. Assert the values in UID order. Return them out of order to verify the index mapping.Also applies to: 680-693
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/chain-live/src/tests.rs` around lines 658 - 659, Update the mock responses in the test around fetch_coldkeys_for_hotkeys to use storage_map_key_account_blake2 with the “SubtensorModule” and “Owner” map, associating distinct coldkeys with the requested hotkeys. Return the mocked Owner entries out of order, then assert the resolved coldkeys by UID order and retain the length assertion.crates/design-http/src/api.rs (1)
1170-1171: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winUse distinct coldkey data in the metagraph fixture.
Line [1171] passes
rawfor both hotkeys and coldkeys. Each test hotkey therefore owns itself. This cannot detect UID-to-owner alignment errors or the required case where two hotkeys share one coldkey.Add a fixture with distinct hotkeys and one shared coldkey. Assert that intake persists the expected
HarnessRow.miner_coldkeyvalues. This targets the PR objective of excluding same-coldkey prior art across rotated hotkeys.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/design-http/src/api.rs` around lines 1170 - 1171, Update the metagraph fixture around the cache.update call to use distinct hotkey data and a separate coldkey fixture with at least two hotkeys sharing one coldkey. Extend the intake test assertions to verify each persisted HarnessRow.miner_coldkey matches the expected shared-owner mapping, covering UID-to-owner alignment and rotated hotkeys.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/db/migrations/0014_miner_coldkey.sql`:
- Around line 9-15: Revise migration 0014_miner_coldkey.sql for the
transactional sqlx::migrate! deployment: add the CHECK constraint as NOT VALID,
create the partial index with CREATE INDEX CONCURRENTLY, and move constraint
validation into a separate non-transactional deployment step using VALIDATE
CONSTRAINT. If the migration framework cannot execute these steps online,
document the required write outage instead.
In `@crates/design-challenge/src/host_sim.rs`:
- Around line 2-7: Update is_prod_env to classify deploy_env value "staging" as
restricted alongside "prod" and "production", while preserving the netuid 100
behavior. Add a regression test confirming staging with a non-100 netuid cannot
enable host simulation through host_sim_allowed.
In `@crates/design-http/src/api.rs`:
- Around line 364-372: In the intake flow around the first metagraph snapshot
that resolves uid, capture and retain coldkey_hex_of(&hotkey) from that same
view. Update the HarnessRow construction to use the retained miner coldkey, and
remove the later st.snapshot() lookup near the miner_coldkey assignment so
persisted identity remains consistent with the uid-producing snapshot.
In `@crates/submission-gating/src/lib.rs`:
- Around line 477-486: Update the key construction logic around the hotkeys and
coldkeys collections so filtering invalid hotkeys preserves their original
source indexes; pair each retained hotkey with the coldkey at that same index
instead of taking the first keys.len() coldkeys. Keep zero-value fallback and
resizing behavior for missing or invalid coldkeys, and add a test covering an
invalid leading hotkey followed by valid paired entries.
In `@docs/DESIGN_CHALLENGE.md`:
- Around line 381-385: Replace the ambiguous same-miner corpus wording with one
consistent phrase meaning “other miners’ prior art only.” Apply the same
corrected phrase in docs/DESIGN_CHALLENGE.md lines 381-385,
docs/DESIGN_CHALLENGE_CHECKLIST.md line 49 (`selfsim_excluded`), and
xtask/src/design_check.rs lines 42-45 (`CONTENT_PINS`).
---
Nitpick comments:
In `@crates/chain-live/src/tests.rs`:
- Around line 658-659: Update the mock responses in the test around
fetch_coldkeys_for_hotkeys to use storage_map_key_account_blake2 with the
“SubtensorModule” and “Owner” map, associating distinct coldkeys with the
requested hotkeys. Return the mocked Owner entries out of order, then assert the
resolved coldkeys by UID order and retain the length assertion.
In `@crates/design-http/src/api.rs`:
- Around line 1170-1171: Update the metagraph fixture around the cache.update
call to use distinct hotkey data and a separate coldkey fixture with at least
two hotkeys sharing one coldkey. Extend the intake test assertions to verify
each persisted HarnessRow.miner_coldkey matches the expected shared-owner
mapping, covering UID-to-owner alignment and rotated hotkeys.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: c5e62321-5700-40b4-beb4-b84bd0feb472
📒 Files selected for processing (40)
bins/design-challenge/tests/resanitize_backfill.rscrates/chain-live/src/lib.rscrates/chain-live/src/storage.rscrates/chain-live/src/tests.rscrates/chain/src/lib.rscrates/challenge-agentic/src/lib.rscrates/challenge-common/src/expected_set.rscrates/db/migrations/0014_miner_coldkey.sqlcrates/db/src/prism_store.rscrates/design-challenge/src/corpus.rscrates/design-challenge/src/host_sim.rscrates/design-challenge/src/score.rscrates/design-challenge/tests/cheat_fixtures.rscrates/design-challenge/tests/e2e_sim.rscrates/design-challenge/tests/orchestrator_retry.rscrates/design-challenge/tests/screenshot_backfill.rscrates/design-db/src/lib.rscrates/design-http/src/api.rscrates/design-http/tests/admin_winners.rscrates/design-store/src/dbstore.rscrates/design-store/src/store.rscrates/prism-challenge/src/agentic.rscrates/prism-challenge/src/api.rscrates/prism-challenge/src/orchestrator.rscrates/prism-challenge/tests/arch_competition.rscrates/prism-challenge/tests/cheat_arch_copy.rscrates/prism-challenge/tests/cheat_metrics.rscrates/prism-challenge/tests/copy_gate.rscrates/prism-challenge/tests/e2e_orchestrate_sim.rscrates/prism-emit/tests/epoch_semantics.rscrates/prism-store/src/dbprism.rscrates/prism-store/src/store.rscrates/submission-gating/src/lib.rsdocs/DESIGN_CHALLENGE.mddocs/DESIGN_CHALLENGE_CHECKLIST.mddocs/PRISM.mddocs/PRISM_RECIPE.mddocs/external-miner/design.mddocs/external-miner/prism.mdxtask/src/design_check.rs
| ALTER TABLE design_harness | ||
| ADD CONSTRAINT design_harness_miner_coldkey_hex | ||
| CHECK (miner_coldkey IS NULL OR miner_coldkey ~ '^[0-9a-f]{64}$'); | ||
|
|
||
| CREATE INDEX ix_design_harness_coldkey | ||
| ON design_harness (miner_coldkey) | ||
| WHERE miner_coldkey IS NOT NULL; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
rg -n 'sqlx::migrate|Migrator|migrate!|no_transaction|no-tx|CREATE INDEX' \
--glob '*.rs' --glob '*.toml' --glob '*.sql' .Repository: BaseIntelligence/base
Length of output: 5550
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- candidate files ---\n'
fd -a 'lib.rs|0014_miner_coldkey.sql' .
printf '\n--- crates/db/src/lib.rs relevant section ---\n'
sed -n '90,115p' crates/db/src/lib.rs
printf '\n--- 0014_miner_coldkey.sql line-numbered ---\n'
cat -n crates/db/migrations/0014_miner_coldkey.sql
printf '\n--- migrations containing BEGIN/COMMIT/LOCK/CREATE INDEX NOT CONCURRENTLY/ALTER TABLE ... VALIDATE CONSTRAINT ---\n'
python3 - <<'PY'
import subprocess, re, pathlib
sql_files = sorted(pathlib.Path('crates/db/migrations').glob('*.sql'))
for f in sql_files:
text = f.read_text()
if any(tok in text.upper() for tok in ['BEGIN WORK', 'BEGIN TRANSACTION', 'CREATE INDEX', 'LOCK TABLE', 'ALTER TABLE', 'NO VALIDATE', 'VALIDATE CONSTRAINT', 'ON DATABASE', 'ON ROLE', 'ROLLBACK']):
print(f'-- {f}')
for i,line in enumerate(text.splitlines(),1):
if re.search(r'(^|\s)(BEGIN WORK|BEGIN TRANSACTION|COMMIT|ROLLBACK|CREATE INDEX|LOCK TABLE|ALTER TABLE|ON TABLE|NOT VALID|VALIDATE CONSTRAINT|ON DATABASE|ON ROLE)\b', line.strip().upper()):
print(f'{i}: {line}')
PY
printf '\n--- repo config for migrations/deploy docs mentioning migration outage/lock/concurrent ---\n'
rg -n 'migration|concurrent|CREATE INDEX|VALIDATE CONSTRAINT|LOCK|outage|CREATE INDEX CONCURRENTLY|sqlx' \
--glob '!target/**' --glob '!node_modules/**' \
README.md docs crates Cargo.toml 2>/dev/null || trueRepository: BaseIntelligence/base
Length of output: 250
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- candidate files ---'
fd 'lib.rs|0014_miner_coldkey.sql' .
printf '%s\n' ''
printf '%s\n' '--- crates/db/src/lib.rs relevant section ---'
sed -n '90,115p' crates/db/src/lib.rs
printf '%s\n' ''
printf '%s\n' '--- 0014_miner_coldkey.sql line-numbered ---'
cat -n crates/db/migrations/0014_miner_coldkey.sql
printf '%s\n' ''
printf '%s\n' '--- migrations containing relevant SQL statements ---'
python3 - <<'PY'
import re
from pathlib import Path
for f in sorted(Path('crates/db/migrations').glob('*.sql')):
text = f.read_text()
tokens = ['BEGIN WORK', 'BEGIN TRANSACTION', 'COMMIT', 'ROLLBACK', 'CREATE INDEX', 'LOCK TABLE', 'ALTER TABLE', 'NOT VALID', 'VALIDATE CONSTRAINT']
if any(tok in text.upper() for tok in tokens):
print(f'-- {f}')
for i, line in enumerate(text.splitlines(), 1):
stripped = line.strip().upper()
if any(tok in stripped for tok in tokens) or re.search(r'ALTER TABLE\s+\w+\s+ADD\s+CONSTRAINT\s+\w+\s+CHECK', stripped) or re.search(r'CREATE\s+INDEX\s+(CONCURRENTLY\s+)?\w+', stripped):
print(f'{i}: {line}')
PY
printf '%s\n' ''
printf '%s\n' '--- deployment/migration guidance mentions ---'
rg -n 'migration|concurrent|CREATE INDEX|VALIDATE CONSTRAINT|LOCK|outage|CREATE INDEX CONCURRENTLY|sqlx|deploy|deployment plan' \
--glob '!target/**' --glob '!node_modules/**' \
README.md docs crates Cargo.toml 2>/dev/null || trueRepository: BaseIntelligence/base
Length of output: 50377
Use an online deployment plan for 0014_miner_coldkey.sql.
sqlx::migrate!("./migrations").run(pool) runs this as a transaction. The added CHECK constraints scan existing rows during ADD CONSTRAINT, and the two new indexes acquire table writes while they build. Split the migration into non-transactional online steps: use NOT VALID for both constraints, add the indexes with CREATE INDEX CONCURRENTLY, then schedule a separate VALIDATE CONSTRAINT step. If that cannot be automated, document the required write outage before deploying this migration.
🧰 Tools
🪛 Squawk (2.61.0)
[warning] 10-11: By default new constraints require a table scan and block writes to the table while that scan occurs. Use NOT VALID with a later VALIDATE CONSTRAINT call.
(constraint-missing-not-valid)
[warning] 13-15: During normal index creation, table updates are blocked, but reads are still allowed. Use concurrently to avoid blocking writes.
(require-concurrent-index-creation)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/db/migrations/0014_miner_coldkey.sql` around lines 9 - 15, Revise
migration 0014_miner_coldkey.sql for the transactional sqlx::migrate!
deployment: add the CHECK constraint as NOT VALID, create the partial index with
CREATE INDEX CONCURRENTLY, and move constraint validation into a separate
non-transactional deployment step using VALIDATE CONSTRAINT. If the migration
framework cannot execute these steps online, document the required write outage
instead.
Source: Linters/SAST tools
| //! Prod/staging must use Docker; host sim needs `BASE_ALLOW_HOST_SIM` + non-prod. | ||
|
|
||
| /// Mainnet netuid / explicit deploy env → prod (host Sim forbidden). | ||
| #[must_use] | ||
| pub fn is_prod_env(netuid: u16, deploy_env: Option<&str>) -> bool { | ||
| if netuid == 100 { | ||
| return true; | ||
| } | ||
| matches!(deploy_env, Some("prod" | "production")) | ||
| netuid == 100 || matches!(deploy_env, Some("prod" | "production")) |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Block host simulation in staging.
When BASE_DEPLOY_ENV is staging and netuid is not 100, is_prod_env returns false. host_sim_allowed then permits host simulation when allow_host_sim is true. Include staging in the restricted-environment predicate and add a regression test.
As per coding guidelines, “Never host Sim in staging or production. Use Docker sandbox there.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/design-challenge/src/host_sim.rs` around lines 2 - 7, Update
is_prod_env to classify deploy_env value "staging" as restricted alongside
"prod" and "production", while preserving the netuid 100 behavior. Add a
regression test confirming staging with a non-100 netuid cannot enable host
simulation through host_sim_allowed.
Source: Coding guidelines
| let miner_coldkey = st | ||
| .metagraph | ||
| .as_ref() | ||
| .and_then(|c| c.snapshot()) | ||
| .and_then(|v| v.coldkey_hex_of(&hotkey)); | ||
| let row = HarnessRow { | ||
| id: id.clone(), | ||
| miner_hotkey: hotkey.clone(), | ||
| miner_coldkey, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Capture miner_coldkey from the membership snapshot.
The intake validates the hotkey with one metagraph snapshot at Line [320]. Lines [364-368] call snapshot() again before persisting the row. A cache refresh between these calls can change the SubtensorModule.Owner value or return no owner.
The persisted identity can then disagree with the snapshot that supplied uid. Capture coldkey_hex_of(&hotkey) from the first view that returned uid, and remove the second snapshot lookup. This identity feeds same-miner corpus exclusion.
Proposed snapshot-consistent capture
let mut uid = None;
+let mut miner_coldkey = None;
if !exists {
if let Some(cache) = &st.metagraph {
match cache.snapshot() {
Some(view) => match view.uid_of_hex(&hotkey) {
- Some(u) => uid = Some(u),
+ Some(u) => {
+ uid = Some(u);
+ miner_coldkey = view.coldkey_hex_of(&hotkey);
+ }
None => {
return json_err(
StatusCode::FORBIDDEN,
@@
- let miner_coldkey = st
- .metagraph
- .as_ref()
- .and_then(|c| c.snapshot())
- .and_then(|v| v.coldkey_hex_of(&hotkey));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/design-http/src/api.rs` around lines 364 - 372, In the intake flow
around the first metagraph snapshot that resolves uid, capture and retain
coldkey_hex_of(&hotkey) from that same view. Update the HarnessRow construction
to use the retained miner coldkey, and remove the later st.snapshot() lookup
near the miner_coldkey assignment so persisted identity remains consistent with
the uid-producing snapshot.
| let keys: Vec<[u8; 32]> = hotkeys | ||
| .iter() | ||
| .filter_map(|h| <[u8; 32]>::try_from(h.as_slice()).ok()) | ||
| .collect(); | ||
| let mut cks: Vec<[u8; 32]> = coldkeys | ||
| .iter() | ||
| .take(keys.len()) | ||
| .map(|c| <[u8; 32]>::try_from(c.as_slice()).unwrap_or([0; 32])) | ||
| .collect(); | ||
| cks.resize(keys.len(), [0; 32]); |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Preserve coldkey source indexes when filtering hotkeys.
keys removes invalid hotkeys, but cks takes the first keys.len() coldkeys. For hotkeys = [invalid, A] and coldkeys = [X, Y], hotkey A receives coldkey X.
Build each retained hotkey and coldkey from the same source index. Add a test with an invalid leading hotkey.
Proposed fix
- let keys: Vec<[u8; 32]> = hotkeys
- .iter()
- .filter_map(|h| <[u8; 32]>::try_from(h.as_slice()).ok())
- .collect();
- let mut cks: Vec<[u8; 32]> = coldkeys
- .iter()
- .take(keys.len())
- .map(|c| <[u8; 32]>::try_from(c.as_slice()).unwrap_or([0; 32]))
- .collect();
- cks.resize(keys.len(), [0; 32]);
+ let mut keys = Vec::with_capacity(hotkeys.len());
+ let mut cks = Vec::with_capacity(hotkeys.len());
+ for (uid, hotkey) in hotkeys.iter().enumerate() {
+ let Ok(hotkey) = <[u8; 32]>::try_from(hotkey.as_slice()) else {
+ continue;
+ };
+ let coldkey = coldkeys
+ .get(uid)
+ .and_then(|coldkey| <[u8; 32]>::try_from(coldkey.as_slice()).ok())
+ .unwrap_or([0; 32]);
+ keys.push(hotkey);
+ cks.push(coldkey);
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| let keys: Vec<[u8; 32]> = hotkeys | |
| .iter() | |
| .filter_map(|h| <[u8; 32]>::try_from(h.as_slice()).ok()) | |
| .collect(); | |
| let mut cks: Vec<[u8; 32]> = coldkeys | |
| .iter() | |
| .take(keys.len()) | |
| .map(|c| <[u8; 32]>::try_from(c.as_slice()).unwrap_or([0; 32])) | |
| .collect(); | |
| cks.resize(keys.len(), [0; 32]); | |
| let mut keys = Vec::with_capacity(hotkeys.len()); | |
| let mut cks = Vec::with_capacity(hotkeys.len()); | |
| for (uid, hotkey) in hotkeys.iter().enumerate() { | |
| let Ok(hotkey) = <[u8; 32]>::try_from(hotkey.as_slice()) else { | |
| continue; | |
| }; | |
| let coldkey = coldkeys | |
| .get(uid) | |
| .and_then(|coldkey| <[u8; 32]>::try_from(coldkey.as_slice()).ok()) | |
| .unwrap_or([0; 32]); | |
| keys.push(hotkey); | |
| cks.push(coldkey); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/submission-gating/src/lib.rs` around lines 477 - 486, Update the key
construction logic around the hotkeys and coldkeys collections so filtering
invalid hotkeys preserves their original source indexes; pair each retained
hotkey with the coldkey at that same index instead of taking the first
keys.len() coldkeys. Keep zero-value fallback and resizing behavior for missing
or invalid coldkeys, and add a test covering an invalid leading hotkey followed
by valid paired entries.
| **other hotkeys' and same-coldkey prior art only** — entries owned by the | ||
| candidate's own `miner_hotkey` **or** `miner_coldkey` are excluded, and so is | ||
| anything created at or after the candidate. After 1-max gating a miner iterates | ||
| via a new hotkey under the same coldkey; those revisions must not be treated as | ||
| cross-miner copies. Selection lives in one place, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Use one unambiguous same-miner corpus marker.
The shared phrase says that same-coldkey prior art is included, while the surrounding contract excludes the candidate's hotkey and coldkey. Keep the specification, checklist, and design-check marker on one corrected phrase.
docs/DESIGN_CHALLENGE.md#L381-L385: replace the ambiguous phrase withother miners' prior art onlyor equivalent wording.docs/DESIGN_CHALLENGE_CHECKLIST.md#L49-L49: updateselfsim_excludedto the same corrected phrase.xtask/src/design_check.rs#L42-L45: updateCONTENT_PINSto require the corrected phrase.
📍 Affects 3 files
docs/DESIGN_CHALLENGE.md#L381-L385(this comment)docs/DESIGN_CHALLENGE_CHECKLIST.md#L49-L49xtask/src/design_check.rs#L42-L45
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/DESIGN_CHALLENGE.md` around lines 381 - 385, Replace the ambiguous
same-miner corpus wording with one consistent phrase meaning “other miners’
prior art only.” Apply the same corrected phrase in docs/DESIGN_CHALLENGE.md
lines 381-385, docs/DESIGN_CHALLENGE_CHECKLIST.md line 49 (`selfsim_excluded`),
and xtask/src/design_check.rs lines 42-45 (`CONTENT_PINS`).
Summary
SubtensorModule.Owneron the cached metagraph (15m TTL) and is persisted at intake (miner_coldkeyondesign_harness/prism_submission, migration 0014).challenge_agentic::same_miner_identity; Designcorpus.rs+ Prism gate / LLM similarity / agentic corpus all use it. Docs + design-check pin updated.Test plan
cargo fmt --check,clippy -D warnings,cargo test --workspacextaskloc-cap, design-check, external-docs-check, consensus-lint, spec-checkDeploy notes
Summary by CodeRabbit
New Features
Behavior Changes
Documentation