docs: clarify train_rows vs full-shard streaming (recipe 1.2) - #5
Conversation
Production advertises train_rows=2048 for the baseline cut; competitive trainers may stream the pinned shard under the 6h/20k-step caps. Note that open 1.4.0/v3 docs PRs are not what prod executes today.
📝 WalkthroughWalkthroughThe getting-started guide now explains dataset row budgets, competitive streaming limits, baseline performance, recipe metadata, and production release status. ChangesDataset Budget Documentation
Estimated code review effort: 1 (Trivial) | ~5 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/getting-started.md`:
- Around line 93-96: Update the production-status wording in the surrounding
recipe-version documentation to replace “Production today” with an explicit
verification date or clearly identify recipe 1.2.0 as the last verified
production value, while preserving the existing live-endpoint guidance.
- Around line 81-83: Update the token-count wording in the getting-started
documentation to use precise notation for an observed quantity, replacing the
mixed approximate/asymptotic expression “~O(10⁹)” with a clear approximate count
or range. Keep the surrounding explanation about loop- and GPU-dependent counts
unchanged.
- Around line 74-75: Update the recipe endpoint references in
getting-started.md, including the “Always trust live...” guidance and the
documented train_rows, val_rows, pin_hex, caps, and 1.2.0 values, to use the
live chain.joinbase.ai gateway with the /challenge/prism/v1/recipe path instead
of /v1/recipe.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1e0b25c3-1767-43b7-8c32-6e478a617091
📒 Files selected for processing (1)
docs/getting-started.md
| | `train_rows` (from `GET /v1/recipe`) | **2048** — baseline / default cut in `ctx` | | ||
| | `val_rows` | **256** — frozen val scored by the harness | |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
curl --fail --silent --show-error \
https://chain.joinbase.ai/v1/recipe |
jq '{version, recipe_version, train_rows, val_rows, caps, pin_hex}'Repository: BaseIntelligence/prism
Length of output: 211
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "Changed docs context:"
sed -n '60,105p' docs/getting-started.md
echo
echo "Locate recipe references:"
rg -n 'v1/recipe|GET /v1/recipe|recipe_version|pin_hex|val_rows|train_rows|Production today' docs README.md examples/baseline/training.py 2>/dev/null || trueRepository: BaseIntelligence/prism
Length of output: 3272
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "$1=$1"
echo "API docs candidates:"
cat -n docs/api.md | sed -n '1,80p'Repository: BaseIntelligence/prism
Length of output: 195
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "docs/api.md route references:"
cat -n docs/api.md | sed -n '1,80p'
echo
echo "All route references in docs and README:"
rg -n 'chain\.joinbase\.ai|/v1/recipe|/recipe/baseline|v1/recipe|recipe version|val_rows|pin_hex' docs README.md 2>/dev/null || trueRepository: BaseIntelligence/prism
Length of output: 4170
Update getting-started.md to document /challenge/prism/v1/recipe.
Docs elsewhere record chain.joinbase.ai as the prod gateway plus the /challenge/prism/... prefix, but these lines and the Always trust live... guidance point to GET /v1/recipe, which is the 404 route. Keep the live URL/path accurate alongside the documented train_rows, val_rows, pin_hex, caps, and 1.2.0 values.
Also applies to: 92-97
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/getting-started.md` around lines 74 - 75, Update the recipe endpoint
references in getting-started.md, including the “Always trust live...” guidance
and the documented train_rows, val_rows, pin_hex, caps, and 1.2.0 values, to use
the live chain.joinbase.ai gateway with the /challenge/prism/v1/recipe path
instead of /v1/recipe.
| fires. Token count then depends on your loop and the GPU — a long Lium run can | ||
| reach ~O(10⁹) tokens. Marketing charts that once said “2.6B tokens · single | ||
| pass” were showing a leader’s **observed** telemetry, not a fixed recipe |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use precise notation for the observed token count.
~O(10⁹) mixes approximation and asymptotic notation. Use a clear expression for an observed quantity.
Proposed wording
-a long Lium run can reach ~O(10⁹) tokens.
+a long Lium run can process on the order of 10⁹ tokens.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fires. Token count then depends on your loop and the GPU — a long Lium run can | |
| reach ~O(10⁹) tokens. Marketing charts that once said “2.6B tokens · single | |
| pass” were showing a leader’s **observed** telemetry, not a fixed recipe | |
| fires. Token count then depends on your loop and the GPU — a long Lium run can | |
| process on the order of 10⁹ tokens. Marketing charts that once said “2.6B tokens · single | |
| pass” were showing a leader’s **observed** telemetry, not a fixed recipe |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/getting-started.md` around lines 81 - 83, Update the token-count wording
in the getting-started documentation to use precise notation for an observed
quantity, replacing the mixed approximate/asymptotic expression “~O(10⁹)” with a
clear approximate count or range. Keep the surrounding explanation about loop-
and GPU-dependent counts unchanged.
| `train_rows` / `val_rows`, recipe version, `pin_hex`). Production today is | ||
| recipe **1.2.0** — open docs PRs that advertise 1.3+/1.4.0/v3 scoring describe | ||
| **unreleased** control-plane work (`prism-better`), not what | ||
| `https://chain.joinbase.ai` executes. `GET /v1/recipe/baseline` returns the |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Replace the relative production date.
Production today will become stale and can contradict the live-endpoint guidance. State the exact verification date, such as As of August 7, 2026, or identify this version as the last verified value.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/getting-started.md` around lines 93 - 96, Update the production-status
wording in the surrounding recipe-version documentation to replace “Production
today” with an explicit verification date or clearly identify recipe 1.2.0 as
the last verified production value, while preserving the existing live-endpoint
guidance.
1 participant
Summary
train_rows: 2048is the baseline cut (~2M GPT-2 tokens), not a hard ceiling for competitive trainers.Test plan
GET https://chain.joinbase.ai/challenge/prism/v1/recipeSummary by CodeRabbit