feat(calc_x): add --val-only to score a checkpoint without training

[Abdelhadi Salmaoui (abdelhadi703)]

Fixes #69.

The Calc-X example can train a model but there is no way to score a checkpoint once the run
is over. AgentLightningRayPPOTrainer.fit already has the exit for it — trainer.val_only
runs one validation pass and returns — but examples/calc_x/train_calc_agent.py never sets
it, and it also pins val_before_train to False, so that exit is unreachable from the
example. That is the missing piece behind "how do I evaluate the trained model".

This adds a --val-only flag:

python train_calc_agent.py --val-only \
    --resume checkpoints/agentlightning/calc_x/global_step_128 \
    --val-file data/test.parquet

--resume goes into trainer.resume_mode and trainer.resume_from_path, which is how verl
restores a checkpoint. --model keeps its earlier meaning, the base model, because the
gateway requires default_proxy.model_name and actor_rollout_ref.model.path to be equal
(docs/25-api-gateway-configuration.md); pointing either of them at a checkpoint makes the
proxy answer "model not found".

verl validates once over the split and exits without an optimizer step. val/reward is the
mean reward over the split, which for Calc-X is the accuracy, since the grader returns 1.0 or
0.0. The run is named calc_x_eval so it does not mix with training runs in W&B.

Worth knowing:

• run_ppo asserts a non-empty train split, so the held-out split is handed in as both. In
  this mode it only sizes the sampler; nothing is trained on it. It therefore needs at least
  data.train_batch_size rows (32 by default). I put that in the docstring and in the docs.
• val_only and val_before_train are re-asserted after the CLI overrides are merged. Those
  merge last, and with the held-out split also serving as the train split, one
  trainer.val_before_train=false on the command line would start optimizing on the data the
  run is supposed to be scored on.
• The checkpoint is only read, never written, so the same split can be scored again later and
  compared across runs.

What I checked, on Windows 10 (AMD64), Python 3.12.13, torch 2.13.0+cpu and VERL 0.8.0, using
the same locked CPU environment CI uses (--group verl-cpu):

• tests/examples/test_calc_x_val_only.py — 6 passed. It pins the val_only / val_before_train
  pair, since setting one without the other silently trains instead of evaluating; the resume
  keys; that the base model is left alone; and a regression test for the override merge.
• The full suite: uv run --locked --no-sync pytest -q tests — 146 passed, 9 failed. All 9 are
  in tests/controller/test_local_reconciler.py and die here with
  module ...local_reconciler.os has no attribute 'killpg'; os.killpg is POSIX-only, so they
  cannot pass on Windows. They fail the same way with this change stashed, so they are not mine.
• python train_calc_agent.py --help lists the flag, and --val-only without --resume exits
  with a clear argparse error rather than silently training.
• ruff check, ruff format --check, scripts/check_headers.py and mkdocs build --strict
  are clean.

I did not run a full evaluation: it needs a GPU and I don't have one on this box. The flag is
wired to the code paths in agentlightning/verl/trainer.py; the rollout itself is untested
here.

An AI assistant helped trace the unreachable val_only exit and draft the change and its tests.

[Copilot]

🟡 Changes recommended

Checkpoints are passed as model directories instead of being restored through VERL’s resume configuration, preventing the documented evaluation flow.

2 open findings

• CLI overrides can disable val-only safety and train on held-out data · New
• Evaluation model name mismatches the gateway model name · New

What changed in this PR

Adds checkpoint-only evaluation for the Calc-X example.

Changes:

• Adds --val-only evaluation mode.
• Adds configuration tests.
• Documents checkpoint evaluation and metrics.

File	Description
examples/​calc_x/​train_calc_agent.py	Adds evaluation CLI and execution path.
tests/​examples/​test_calc_x_val_only.py	Tests evaluation configuration.
docs/​50-example-calc-x.md	Documents checkpoint scoring.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Abdelhadi Salmaoui (abdelhadi703)]

Both addressed in 3d00516.

On the first one you are right that the overrides merge last. val_only, val_before_train, resume_mode and resume_from_path are now re-asserted after the merge, so --val-only trainer.val_before_train=false no longer gets through. A regression test passes exactly those two overrides and asserts the mode still holds.

On the second I went a bit further than the model name, because aiming actor_rollout_ref.model.path at the checkpoint was the wrong lever in another way too: trainer.resume_mode stayed at its auto default, so _load_checkpoint looked in default_local_dir -- empty on a fresh eval run -- printed "Training from scratch" and would have scored an untrained model without saying so.

The checkpoint now travels through trainer.resume_mode=resume_path / trainer.resume_from_path, which is what verl restores from. --model keeps its earlier meaning, so the gateway still finds its model name and run_local.sh works as documented.

The flag is --resume <.../global_step_N> now; docs and tests follow. 6 tests pass, 146 passed on the full suite (the 9 failures are pre-existing os.killpg tests that cannot run on Windows).