fix: microduck-walk-flat standing regression — close-feet yaw sign, gait-phase pinning, penalty_scale metric

[wlgys8]

问题

microduck-walk-flat 在 #52（direct → manager 迁移）之后，FastSAC 训练稳定收敛到"站立不动"局部最优：tracking_lin_vel 卡在 ~0.06、feet_phase 卡在 ~0.117（正常行走 ~0.085 / ~0.149）。同 batch 同 seed 下 direct env（fe3dd90）14/14 训练成功，manager env 0/8 全部失败。

根因：penalty_close_feet_xy_reward yaw 符号反向

迁移时把旧实现 rotate_vector(q, (1,0,0))（base 前向在世界系的 heading）换成了 rotate_inverse_components(q, (1,0,0))——后者是世界 x 轴在 base 系的表示，yaw 反号。合成 input/output fuzz 显示：99.4% 的 (yaw, Δx, Δy) 样本横向距离算错，threshold=0.05 的二值判定 31% 相反。行走步态（双脚前后错开 + 任意 yaw 偏差/转弯）被系统性误判为"双脚交叉"并施以 weight −10 的惩罚，策略收敛到站立不动。这同时解释了 ang_vel 追踪正常（原地转身双脚对称，符号错误不触发）。

修复：把世界系脚距向量逆旋转回 base 系直接取横向分量（纯 yaw 下与旧实现 8000 样本逐位相等，倾斜时语义更正确）。

附带修复

• _lane_phase 只钉死左脚时钟：stand 指令下旧实现 phase[stand] = π 钉死双脚，manager 版只钉 lane 0，右脚参考轨迹继续振荡，20% 样本携带矛盾梯度。现两条 lane 一起钉死，并简化为循环形式。
• penalty_scale 指标丢失：旧 direct env 每步导出 metrics/penalty_scale，迁移时丢失。现通过 WalkCommand.reset() 的 ctx.metrics 恢复导出。

验证

修复后同 seed=1、batch=2048（此前 0/8 的配置）：

指标	修复前	修复后	fe3dd90 direct 基线
tracking_lin_vel	0.058–0.061	0.081	0.081–0.086
feet_phase	0.116–0.117	0.150	0.149
penalty_close_feet_xy	卡在 −0.015	0.000	≈0
mean_return	318–377	392	394

motrix_envs/tests/test_humanoid_walk.py 18/18 通过。

[Copilot]

🟡 Changes recommended

penalty_scale is currently written to ctx.metrics before the curriculum update in reset(), which makes the exported metric lag behind the value actually used after the update.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR fixes a post-migration training regression in the manager-based humanoid walk task (microduck-walk-flat) by correcting the close-feet penalty’s frame handling, stabilizing gait-phase behavior under standing commands, and restoring the penalty_scale metric export used for curriculum/diagnostics.

Changes:

• Fix penalty_close_feet_xy_reward to compute lateral separation by inverse-rotating the foot delta into the base frame (avoids yaw sign inversion).
• Pin both gait-phase lanes to π during standing commands to prevent contradictory phase gradients.
• Re-export penalty_scale via WalkCommand.reset() metrics.

File summaries

File	Description
motrix_envs/src/motrix_envs/locomotion/humanoid/walk_manager_mdp/rewards.py	Corrects close-feet lateral distance computation to avoid yaw sign inversion regression.
motrix_envs/src/motrix_envs/locomotion/humanoid/walk_manager_mdp/command.py	Pins both gait phase lanes on stand and restores penalty_scale metric export.

Review details

Suppressed comments (1)

motrix_envs/src/motrix_envs/locomotion/humanoid/walk_manager_mdp/command.py:126

• ctx.metrics["penalty_scale"] is written before the curriculum update logic, so when an episode ends and penalty_scale is adjusted, the exported metric lags by one reset and doesn’t reflect the post-update scale used for subsequent steps.

        ctx.metrics["penalty_scale"] = float(self.penalty_scale[0])
        if not self.curriculum_enabled or ctx.env_ids.size == 0:
            return
        ep_len = self.steps[ctx.env_ids, 0].astype(np.float64) + 1.0
        self.avg_ep_len[0] = np.float32(0.99 * self.avg_ep_len[0] + 0.01 * float(ep_len.mean()))
        if self.avg_ep_len[0] < self.level_down_threshold:
            self.penalty_scale[0] *= np.float32(1.0 - self.degree)
        elif self.avg_ep_len[0] > self.level_up_threshold:
            self.penalty_scale[0] *= np.float32(1.0 + self.degree)
        self.penalty_scale[0] = np.float32(
            min(max(float(self.penalty_scale[0]), float(self.min_scale)), float(self.max_scale))
        )

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wlgys8]

Both review comments addressed in 74d16b9: docstring now says "both lanes' phase clocks", and the close-feet comment clarifies the z=0 (XY-only, vertical separation intentionally ignored) delta rotation.