docs: sync AutoModel recipe 2.0 miner guide - #9
Open
echobt wants to merge 1 commit into
Open
Conversation
Update public miner docs for live recipe 2.0 (automodel@v0.5.0 pin + patch, BYOK X-Lium-Api-Key) and retire the architecture.py/training.py path.
📝 WalkthroughWalkthroughThe documentation replaces the legacy two-script workflow with recipe 2.0.0 AutoModel patch submissions. It updates packaging, authentication, API operations, troubleshooting, evaluation, scoring, emission, and publication guidance. ChangesAutoModel Patch Workflow
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
docs/submit.md (1)
32-40: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winMake miner identity explicit in both JSON examples.
docs/submit.md#L32-L40: ShowX-Miner-Hotkeyor document the requiredminer_hotkeyfield insubmission.json.docs/prism.md#L85-L89: Apply the same identity requirement to the full-guide JSON command.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/submit.md` around lines 32 - 40, Make miner identity explicit in the JSON submission examples: update docs/submit.md lines 32-40 and docs/prism.md lines 85-89 to show the X-Miner-Hotkey header or document the required miner_hotkey field in submission.json, using the same approach in both locations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/getting-started.md`:
- Around line 49-54: Update the telemetry-hook contract in
docs/getting-started.md lines 49-54, docs/prism.md lines 49-52,
docs/troubleshooting.md lines 12-13, and examples/baseline/README.md lines 27-28
so prism_telemetry.report(...) remains required while finish_evaluation() is
explicitly optional when evaluation terminates early; keep all four documents
consistent.
In `@docs/prism.md`:
- Around line 77-83: Define one shared gateway variable before the first curl
example in docs/prism.md:77-83 and before the recipe request in
docs/getting-started.md:80-88; reuse that variable for recipe retrieval in
docs/prism.md:98-102 and replace the inconsistent $GATEWAY reference in
docs/prism.md:157-161, keeping the variable name consistent across all affected
examples.
In `@README.md`:
- Around line 58-60: Update the packaging instructions in README.md lines 58-60
and docs/submit.md lines 21-23 to include a conditional step that adds
prism.toml to submission.zip when the file is used, while preserving the
existing automodel.base and automodel.patch packaging workflow.
- Around line 34-39: Replace the cleartext staging gateway documentation so
credential-bearing requests never use HTTP: update README.md lines 34-39,
docs/submit.md lines 109-116, and docs/api.md lines 3-6 to use an HTTPS staging
base URL or explicitly separate staging authentication from the live
X-Lium-Api-Key/BYOK workflow, keeping all three references consistent.
---
Nitpick comments:
In `@docs/submit.md`:
- Around line 32-40: Make miner identity explicit in the JSON submission
examples: update docs/submit.md lines 32-40 and docs/prism.md lines 85-89 to
show the X-Miner-Hotkey header or document the required miner_hotkey field in
submission.json, using the same approach in both locations.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 4bf733d0-f924-448d-a50d-6b64700ceb58
📒 Files selected for processing (9)
README.mddocs/README.mddocs/api.mddocs/getting-started.mddocs/prism.mddocs/scoring.mddocs/submit.mddocs/troubleshooting.mdexamples/baseline/README.md
Comment on lines
+49
to
+54
| ## Telemetry hooks (still required) | ||
|
|
||
| Instead of shipping both scripts you can submit `training.py` + `arch_id` referencing a | ||
| **published** architecture. The master pulls `architecture.py` from the registry; the | ||
| same harness contract applies unchanged. Published archs: `GET /v1/architectures`. | ||
| See [Submit](submit.md#training-only-entries). | ||
| The harness wrap still requires `prism_telemetry` reporting / | ||
| `finish_evaluation` under the AutoModel train entry. Patches that remove or | ||
| bypass those hooks fail review (`missing_telemetry_hooks`, zero score, | ||
| terminal). |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- affected documentation ---'
for f in docs/getting-started.md docs/prism.md docs/troubleshooting.md examples/baseline/README.md; do
echo "### $f"
sed -n '1,180p' "$f" | nl -ba | sed -n '1,180p'
done
printf '%s\n' '--- telemetry references ---'
rg -n -i -C 3 'prism_telemetry|finish_evaluation|missing_telemetry_hooks|AutoModel|telemetry' . \
-g '!node_modules' -g '!dist' -g '!build'
printf '%s\n' '--- repository files ---'
git ls-files | sed -n '1,240p'Repository: BaseIntelligence/prism
Length of output: 30008
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- exact affected sections ---'
for spec in \
'docs/getting-started.md:45:56' \
'docs/prism.md:43:56' \
'docs/troubleshooting.md:1:15' \
'examples/baseline/README.md:1:30' \
'examples/baseline/training.py:1:100' \
'top-model/training.py:1:96' \
'docs/scoring.md:59:70'; do
file=${spec%%:*}; rest=${spec#*:}; start=${rest%%:*}; end=${rest##*:}
echo "### $file:$start-$end"
sed -n "${start},${end}p" "$file" | awk '{printf "%6d %s\n", NR + '"$start"' - 1, $0}'
done
printf '%s\n' '--- all exact telemetry contract statements ---'
rg -n -C 2 'required|optional|either way|finish_evaluation|missing_telemetry_hooks' \
docs examples top-model README.md
printf '%s\n' '--- executable repository/config references to the error name ---'
rg -n 'missing_telemetry_hooks|telemetry_hooks|finish_evaluation|prism_telemetry' \
.github docs examples top-model README.mdRepository: BaseIntelligence/prism
Length of output: 24646
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
import ast
from pathlib import Path
docs = [
"docs/getting-started.md",
"docs/prism.md",
"docs/troubleshooting.md",
"examples/baseline/README.md",
]
terms = ("prism_telemetry", "finish_evaluation", "missing_telemetry_hooks")
for name in docs:
text = Path(name).read_text()
hits = []
for i, line in enumerate(text.splitlines(), 1):
if any(term in line for term in terms):
hits.append((i, line.strip()))
print(f"### {name}")
for line_no, line in hits:
print(f"{line_no}: {line}")
for name in ("examples/baseline/training.py", "top-model/training.py"):
tree = ast.parse(Path(name).read_text(), filename=name)
train = next(
node for node in tree.body
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == "train"
)
calls = []
for node in ast.walk(train):
if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute):
if isinstance(node.func.value, ast.Name) and node.func.value.id == "prism_telemetry":
calls.append(node.func.attr)
print(f"### {name} train() telemetry calls")
print(", ".join(calls) or "(none)")
PYRepository: BaseIntelligence/prism
Length of output: 1366
Use one authoritative telemetry-hook contract.
Keep prism_telemetry.report(...) required. Mark finish_evaluation() as optional for early evaluation termination in the three conflicting locations.
📍 Affects 4 files
docs/getting-started.md#L49-L54(this comment)docs/prism.md#L49-L52docs/troubleshooting.md#L12-L13examples/baseline/README.md#L27-L28
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/getting-started.md` around lines 49 - 54, Update the telemetry-hook
contract in docs/getting-started.md lines 49-54, docs/prism.md lines 49-52,
docs/troubleshooting.md lines 12-13, and examples/baseline/README.md lines 27-28
so prism_telemetry.report(...) remains required while finish_evaluation() is
explicitly optional when evaluation terminates early; keep all four documents
consistent.
Comment on lines
+77
to
+83
| ```bash | ||
| # ZIP via gateway (preferred) | ||
| curl -sS -X POST "$BASE_GATEWAY/challenge/prism/v1/submissions" \ | ||
| -H 'content-type: application/zip' \ | ||
| -H "X-Miner-Hotkey: <64 lowercase hex>" \ | ||
| -H "X-Lium-Api-Key: $LIUM_API_KEY" \ | ||
| --data-binary @submission.zip |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Define and reuse one gateway variable.
docs/prism.md#L77-L83: DefineGATEWAYorBASE_GATEWAYbefore the first curl command.docs/getting-started.md#L80-L88: DefineGATEWAYbefore the recipe request.docs/prism.md#L98-L102: Use the same defined variable for recipe retrieval.docs/prism.md#L157-L161: Replace the inconsistent$GATEWAYreference in the precheck command.
📍 Affects 2 files
docs/prism.md#L77-L83(this comment)docs/getting-started.md#L80-L88docs/prism.md#L98-L102docs/prism.md#L157-L161
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/prism.md` around lines 77 - 83, Define one shared gateway variable
before the first curl example in docs/prism.md:77-83 and before the recipe
request in docs/getting-started.md:80-88; reuse that variable for recipe
retrieval in docs/prism.md:98-102 and replace the inconsistent $GATEWAY
reference in docs/prism.md:157-161, keeping the variable name consistent across
all affected examples.
Comment on lines
34
to
+39
| | Challenge id | `prism` | | ||
| | Production gateway | `https://chain.joinbase.ai` | | ||
| | Staging gateway | `http://staging.api.joinbase.ai` | | ||
| | Submit path | `/challenge/prism/v1/submissions` | | ||
| | Recipe | v1.2.0 — telemetry hooks required | | ||
| | Recipe | **2.0.0** — AutoModel pin + patch (`automodel@v0.5.0`) | | ||
| | Live GPU | Miner-funded Lium — pass `X-Lium-Api-Key` | |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Do not publish a cleartext staging gateway for credential-bearing requests.
README.md#L34-L39: Replace the HTTP staging endpoint or document a staging flow that never sends a realX-Lium-Api-Key.docs/submit.md#L109-L116: Use an HTTPS staging base URL or separate staging authentication from the live BYOK workflow.docs/api.md#L3-L6: Update the staging gateway documentation so clients do not reuse credential-bearing commands over HTTP.
📍 Affects 3 files
README.md#L34-L39(this comment)docs/submit.md#L109-L116docs/api.md#L3-L6
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 34 - 39, Replace the cleartext staging gateway
documentation so credential-bearing requests never use HTTP: update README.md
lines 34-39, docs/submit.md lines 109-116, and docs/api.md lines 3-6 to use an
HTTPS staging base URL or explicitly separate staging authentication from the
live X-Lium-Api-Key/BYOK workflow, keeping all three references consistent.
Comment on lines
+58
to
60
| # After forking the pin and producing automodel.base + automodel.patch: | ||
| zip -j submission.zip automodel.base automodel.patch # + prism.toml if used | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Ensure the documented package command includes optional prism.toml.
README.md#L58-L60: Add a conditional ZIP update forprism.toml.docs/submit.md#L21-L23: Add the same conditional packaging step to the preferred submission workflow.
📍 Affects 2 files
README.md#L58-L60(this comment)docs/submit.md#L21-L23
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 58 - 60, Update the packaging instructions in
README.md lines 58-60 and docs/submit.md lines 21-23 to include a conditional
step that adds prism.toml to submission.zip when the file is used, while
preserving the existing automodel.base and automodel.patch packaging workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
git diff→ submitautomodel.base+automodel.patch(+ optionalprism.toml), live pin idautomodel@v0.5.0.X-Lium-Api-Key, patch apply /unsupported_layout/recipe_versiontroubleshooting, and retire the livearchitecture.py/training.pypath (examples/baseline marked historical).assets/branding; docs-only — no control-plane source.Paired with control-plane work: BaseIntelligence/base#120
Test plan
/v1/recipe(automodel@v0.5.0)X-Lium-Api-Keyunsupported_layout,recipe_version, patch apply failuresexamples/baseline/clearly warns it is rejected on live 2.0Summary by CodeRabbit