pollen-robotics/microduck       # 机器人软件 + SDK：Rust workspace，22 个成员 crate
pollen-robotics/microduck_rl    # 强化学习训练：Python/mjlab（MuJoCo Warp）+ rsl_rl PPO

RL training environments for Microduck — a ~800 g, ~25 cm tall bipedal
robot with 14 Dynamixel XL330 servos — built on mjlab
(MuJoCo Warp) with PPO (rsl_rl). Policies are trained here at 50 Hz, exported to
ONNX, and deployed by the runtime in the `pollen-robotics/microduck` repo on
the real robot. Sim2real transfer is the whole point: every convention below
exists because breaking it produced a policy that worked in the viewer and
failed on hardware.

# Workspace root for the robot daemon.
#
# One crate per service or tool, matching the service split in
# docs/design/architecture.md §1. `robotd`, `btd` and `mediad` are all siblings now.
[workspace]
resolver = "3"
members = ["btd", "configd", "duck-control", "duck-detect", "duck-ether", "duck-ipc-proto",
  "duckctl", "kinematics", "mediad", "odometry", "pad-imu", "padd", "pet-detect", "sounds",
  "tof", "updater", "robotctl", "robotd", "robotd-params", "test-support", "uyvy", "xtask"]

# ONNX Runtime, which robotd dlopens to run a policy. One source of truth: `xtask package`
# bakes these into the release's preinstall hook, and a test asserts scripts/setup-board.sh
# agrees with them. Two copies drifting apart is exactly how a board ended up with 1.20.1
# against an `ort` that panics below 1.23.
[workspace.metadata.onnxruntime]
floor  = "1.23"
target = "1.28.0"

# The official policy set, which `robotd` runs and which lives on the Hugging Face Hub rather
# than in this repository — a gait retrain should not need a daemon release, and a daemon fix
# should not re-ship six megabytes of unchanged weights.
[workspace.metadata.policies]
repo    = "pollen-robotics/microduck-policies"
version = "v5"

//! The observation vector the policy sees.
//!
//! **This is the highest-risk code in the crate.** It is a flat array of 61 floats whose
//! every index must match what the policy was trained against. A wrong offset does not fail
//! loudly — it produces a plausible-looking robot that falls over, and the symptom looks
//! like a tuning or timing problem rather than an indexing one.

index   width  contents
0..3        3  gyro, trunk frame, rad/s
3..6        3  projected gravity, trunk frame, unit vector
6..20      14  joint position minus home pose, mouth excluded
20..34     14  joint velocity, mouth excluded
34..48     14  previous action, mouth excluded
48..61     13  command (below)

# Everything except `duckctl`, and that exception is the whole reason this key exists.
#
# `cargo board --bins` — in `dev-push.sh` and in the release workflow — builds every default
# member for aarch64, which is right for a robot's daemons and wrong for a client a developer
# runs on their own machine: it would cross-compile a Bluetooth stack for a board that must never
# see one, on the release path, for nothing.
#
# The alternative was naming the binaries explicitly at each `--bins` call site. There are two of
# them and they would have to be kept in step by hand, which is the failure this repo keeps
# writing down. One list here, and a new daemon is picked up by both without anybody remembering.

## Docs own mechanisms; one page each

`docs/README.md` assigns every mechanism to one design doc. When a fact belongs to a page listed
there, every other page says one sentence and links. When two pages disagree, the one that does
not own the mechanism is the bug — and when behaviour and a design doc disagree, the doc is
the bug.

app-path-design.md     architecture.md        boot-recovery-net.md
policy-channel-design.md  remote-access-design.md  remote-webrtc.md
restart-order.md       robotd-design.md       simulation.md
updater-design.md      webrtc-console.md

## A consumer uses WebRTC. `media.stream` is the fallback

The robot publishes H.264 over WebRTC, and that is the default for anything consuming a duck's
camera — it is encrypted end to end, it carries the control channel on the same session, and it
has a return path. ...

This is worth stating because the repository reads the other way round if you only follow the
code: `media.stream` was built when the relay endpoint was dead and WebRTC genuinely could not
connect from a data centre, so its module doc argues its own case at length. That endpoint is
fixed. Do not conclude from the volume of prose that it is the preferred path.

/// A method together with its parameters.
///
/// Every request is built from one of these and read back as one, so a method can never be
/// paired with another method's parameters — the drift this crate exists to prevent.
#[derive(Debug, Clone, PartialEq)]
pub enum Call {
    /// Version handshake. The first call on a connection.
    Hello(HelloParams),
    ...

    // ── intents ──────────────────────────────────────────────────────────────
    /// Continuous. Send as a notification.
    RobotMove(MoveParams),
    /// Continuous. Send as a notification.
    RobotHead(HeadParams),
    /// Discrete. Send as a request; the answer is [`LookResult`].
    RobotLook(LookParams),
    RobotStop,
    RobotEnable(EnableParams),
    /// Power the joints and ramp to the home pose. No policy needed.
    RobotInit,
    /// Cut power to the joints. The robot collapses if nothing holds it.
    RobotRelax,
    /// Reboot servos (all of them, or the ids named), then limp. See [`method::ROBOT_REBOOT_MOTORS`].
    RobotRebootMotors(RebootMotorsParams),
    /// Run a one-shot skill, or toggle sit↔stand.
    RobotDo(DoParams),
    /// Standing body pose. Continuous. Send as a notification.
    RobotPose(PoseParams),
    /// Mouth opening. Continuous. Send as a notification.
    RobotMouth(MouthParams),

/// Request identifier. `None` on a [`Request`] makes it a notification.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum Id {
    Number(u64),
    Text(String),
}

/// The service that owns the answer to a call.
///
/// One socket per service, connected directly — there is no broker (`architecture.md` §2.2). A
/// transport adapter holds connections to the services whose calls it carries, and to no others:
/// `btd` holds three, and `padd` being absent from them is deliberate rather than incidental —
/// `padd` is the unprivileged client whose whole value is having no special access.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Service {
    /// `updaterd`, at [`DEFAULT_SOCKET`].
    ...

    /// Read the pairing PIN.
    ///
    /// Exists so `btd` can answer a BLE passkey request without owning config. It must never be
    /// routed to BLE — a PIN an unpaired peer can read authorises nothing — and `btd`'s routing
    /// table has a test saying so.
    SystemPairingPin,

    /// Prove knowledge of the robot's pairing PIN.
    ///
    /// Answered by the **transport** rather than by any service, which makes it unlike every
    /// other call here. BLE cannot express a fixed, printed-on-the-robot passkey — the spec has
    /// the *displaying* side generate a random one, and a headless robot can display nothing — so
    /// the PIN check moved from the link layer to this one, where we define the rules. See
    /// `docs/design/app-path-design.md` §5.
    SystemAuthenticate(AuthenticateParams),

    /// `btd` subscribing to what it should advertise. Answered, then a stream of
    /// [`method::CHORALE_BEACON`] notifications.
    ChoraleSubscribe,
    /// `robotd` telling `btd` what to advertise. A notification.
    ChoraleBeaconSet(ChoraleAdvertise),
    /// `btd` telling `robotd` what it heard. A notification.
    ChoraleHeard(ChoraleHeard),

//! Turning sensors and a command into joint targets — and scheduling the skills.
//!
//! Everything here is pure computation between [`duck_control::io::RobotIo::read`] and the
//! safety layer's `apply`. It holds no IO handle — by construction it cannot command a
//! motor, only propose targets.
//!
//! The tick, in order:
//!
//! ```text
//! skill windows ← advance / expire (roulade window, kick timer, ground-pick phase, sit↔stand rise)
//! command      ← the caller's smoothed command, re-encoded for the active skill
//! net          ← roulade > kick > ground pick > sit/rise > stand-by-magnitude > walk
//! action       ← ONNX
//! targets      ← home pose + action_scale × action
//! filters      ← optional first-order low-pass on head and legs
//! ```

pub struct Tuning {
    /// Scales raw policy output before it becomes a joint offset. The prototype's current
    /// alpha default.
    pub action_scale: f64,
    /// The standing policy is trained to be applied whole.
    pub standing_action_scale: f64,
    /// Standing runs softer, at this fraction of the running gain. `--standing-kp-ratio`.
    pub standing_gain_ratio: f64,
    pub gain: u16,
    /// First-order low-pass on the head joints. `None` is no filtering. The alpha policies
    /// are trained with 0.5 — it must match training or transfer degrades.
    pub head_lowpass: Option<f64>,
    /// Same, for the ten leg joints. Trained with 0.7.
    pub legs_lowpass: Option<f64>,
}

impl Default for Tuning {
    fn default() -> Self {
        Self {
            action_scale: 0.9,
            standing_action_scale: 1.0,
            standing_gain_ratio: 0.8,
            gain: 200,
            head_lowpass: Some(0.5),
            legs_lowpass: Some(0.7),
        }
    }
}

        } else if let Some(phase) = self.ground_pick {
            // The twist slots carry the phase encoding; head and body are zero-padded,
            // mirroring the training env's `zero_command_padding`.
            let angle = std::f64::consts::TAU * phase;
            let c = Command {
                twist: [angle.cos(), angle.sin(), 0.0],
                ..Command::default()
            };
            (Net::GroundPick, c, "ground_pick".into())
        } else {
            let mut c = *command;
            match self.sit {
                // The posture flag rides the twist vx slot: 1 = sit, 0 = stand. Head and
                // body slots stay live — the prototype keeps them in the buffer too.
                Sit::Sitting => {
                    c.twist = [1.0, 0.0, 0.0];
                    (Net::SitStand, c, "sit".into())
                }

//!  - **A kick window runs at standing tuning.** The kick's observation carries an all-zero
//!    command, and in the prototype the standing transition fires on exactly that — so a
//!    kick runs at `standing_action_scale` and the softened standing gain. Kept, because
//!    the kicks were tuned against it.

/// **One dropped Dynamixel transaction is ordinary** — `robotd` says so itself when it logs
/// one — and it must therefore be *invisible*. It was not: a failed read used to stop the
/// policy for that tick, which commanded the hold pose and reset the controller, so every
// ordinary dropped read produced a visible twitch. Measured on a bench robot at ~8 drops a
/// minute with a monitor attached, which is exactly the reported "random tiny spasms".
///
/// Coasting is safe because the observation is *already* a tick old by construction: at
/// 50 Hz the policy is trained on data of exactly this age, and a second tick of it is
/// inside that. Three ticks (60 ms) covers a drop, a retry and a slow tick; past that the
/// robot genuinely cannot see, and holding still is the honest answer.
const COAST_TICKS: u32 = 3;

//!  1. **Body x, y and yaw are hardcoded zero.** They are unbound in the training
//!     environment, so an all-zero body command is the *nominal* encoding, not a
//!     placeholder standing in for something better.
//!  2. **Head targets ride in the command, and are not added on top of the policy output.**
//!     The prototype does both, in different modes, and gates the post-hoc addition behind
//!     `if !new_cmd_obs` with the note "head_offsets are a COMMAND fed via the obs vector
//!     instead — don't double-add it here". Doing both would bend the head twice.

    // ── update.* ─────────────────────────────────────────────────────────────
    Check(ComponentParams),
    Apply(ApplyParams),
    Rollback(ComponentParams),
    ResetToGolden(ComponentParams),
    Select(SelectParams),
    Pin(PinParams),
    Status,
    ListInstalled(ComponentParams),
    Log(LogParams),
    /// One run's full transcript. `update.log` says which runs exist; this says what one did.
    Show(ShowParams),
    /// Turns the connection into a stream of [`method::PROGRESS`] notifications.
    Subscribe,

//! The 50 Hz control loop, the `robot.*` socket, and the health the updater gates on

    // ── detector.* ───────────────────────────────────────────────────────────
    /// What detector is installed and what the Hub offers; see [`method::DETECTOR_CHECK`].
    DetectorCheck,
    /// Install a detector and restart `mediad` onto it.
    DetectorInstall(PolicyInstallParams),

    /// Subscribe to the raw pad input stream. Answered by `padd`, not `configd`.
    PadInput,
    /// Subscribe to the ToF depth stream. Answered by `tofd`.
    TofStream,
    /// Subscribe to the head IMU (BMI088 on the HAT); see [`method::HEAD_IMU_STREAM`].
    HeadImuStream,

    /// Play a voice-bank sound.
    RobotSound(SoundParams),
    /// Pick the ToF theremin up or put it down. Discrete; the answer is [`ThereminResult`].
    RobotTheremin(ThereminParams),

//! MJCF-driven forward kinematics, compiled for the control loop.
//!
//! The prototype's `microduck_kinematics` crate proved the approach — parse the
//! training MJCF, walk the tree — but its query path was built for convenience:
//! joint angles travelled in a `HashMap<String, f64>`, and asking for one site
//! recomputed every body in the robot into a freshly allocated `Vec`. Fine at
//! bench cadence; wasteful inside a 50 Hz loop that asks for both feet every
//! tick.
//!
//! Here the model is *compiled* once at load: names are resolved to indices,
//! and each site gets its own flattened root→site chain of links. A query is
//! then a fold over that chain — no hashing, no allocation, no bodies the site
//! does not hang from. Angles are a plain `&[f64]` indexed by [`Model`]'s joint
//! order; resolve names to indices once with [`Model::joint_index`], not per
//! query.
//!
//! Correctness is pinned two ways: `tests/fk_against_mujoco.rs` compares every
//! site against MuJoCo's own `mj_kinematics` on 64 random poses, and
//! [`head`] pins the sign conventions the rest of the system steers by.

#!/bin/sh
# A duck in MuJoCo, driven by the real daemon, in one command.
#
#   scripts/duck-sim              # a window opens, the duck stands up, and it is yours
#   scripts/duck-sim drive        # walk it forward for a few seconds
#   scripts/duck-sim ctl health   # anything robotctl does
#   scripts/duck-sim down
#
# And the same duck, as a machine you log into:
#
#   scripts/duck-sim boot         # the daemons under real systemd, in a container
#   scripts/duck-sim shell        # you are on the duck
#   duck-a # robotctl health
#
# `boot` needs sudo, because `systemd-nspawn` does. What it buys over the plain form is the part
# that is not fakeable: the real unit files, with their real User=, groups, RuntimeDirectory= and
# hardening, under a real init — which is what makes `robotctl update apply` and the health gate
# behave here the way they behave on a robot.

**Status:** built and in use. `robotd --sim`, `tofd --sim` and `mediad --sim-camera` are here, with
`microduck_rl`'s `duck-body` serving the other half; the containers (§8), the ether (§5) and the
per-duck cameras all run from `scripts/duck-sim`. Measured: the daemon holds `50.0 of 50.0 Hz · 0
missed` against a MuJoCo body, detects a seated boot from the simulator's own joint angles, and
`robotctl robot init` runs the sitstand policy until the duck is upright and stays there; four ducks
in containers sing a full chorale over the ether. ...

The goal is a duck you develop against exactly as you develop against a robot: the same binaries,
the same units, the same `robotctl`, the same `duckctl open` — with the body in MuJoCo instead of on
the desk. Not a mock, and not a test harness. A twin, with a written-down boundary.

# The three things this exists to stop anyone typing by hand, because none of them is guessable:
#
#   * policies live at /opt/robot/daemon/current on a robot and in this repo on a laptop, so every
#     one has to be named in a params file;
#   * `ort` dlopens libonnxruntime and a laptop has no system one — the RL repo's venv does;
#   * a unix socket path is capped at ~108 bytes, so the state directory has to be short.

## Commands

```bash
uv run list-envs                                    # live task registry
uv run train <TASK_ID> --env.scene.num-envs 4096    # train (add --hf-jobs for Hugging Face Jobs)
uv run train <TASK_ID> --env.scene.num-envs 64 --agent.max_iterations 5   # SMOKE TEST — always run first
uv run play <TASK_ID> --wandb-run-path <entity/project/run_id>
uv run scripts/export.py <TASK_ID> --wandb-run-path <...>   # → ONNX (bakes obs normalizer — mandatory path)
uv run publish --task <TASK_ID> --wandb-run-path <...> --checkpoint N --repo <user>/microduck-<name> --kind episodic --duration-s 4.0
                                                    # → HF Hub repo (policy.onnx + schema-2 manifest.json + README) the daemon loads via `robotctl policy add`
uv run scripts/infer_policy.py --walking out.onnx   # CPU MuJoCo deployment rehearsal (BAM M6 actuators as in training; --no-bam = XML PD)
uv run --with pytest pytest tests/

划线句：**5 迭代 × 64 环境的冒烟测试，花几分钱抓住约 95% 的配置错误；不冒烟不跑长训**。RL 实验最大的浪费不是 GPU 时费，是 90 分钟后才发现奖励项拼错字的一天。命令表还埋着完整链路的顺序：`train → play（回看）→ export（唯一合法转 ONNX 的路）→ infer_policy（CPU 部署彩排）→ publish（Hub）`，最后由机器人侧 `robotctl policy add` 消费——第 10 章逐环展开。

`infer_policy.py` 的注释点出一个细节：彩排用的执行器是 **BAM M6，与训练一致**（`--no-bam` 才退回 XML 自带 PD）——连"验证部署"这一步都在守护执行器保真度契约。

## 7.3 代码走读：仓库地图——一个 cfg 一个任务族

`AGENTS.md` 的 Repo map 节（节选）：

```markdown
- `src/mjlab_microduck/tasks/mdp.py` — ALL custom MDP functions (rewards, events,
  observations, commands, curricula). Add new functions here, grouped by task.
- `src/mjlab_microduck/tasks/microduck_*_env_cfg.py` — one cfg module per task
  family. `microduck_velocity_env_cfg.py` is the main walking recipe AND the
  shared base (robot, DR, obs, commands) other envs build on or mirror.
- `src/mjlab_microduck/tasks/__init__.py` — task registration (base + `-Backlash-` variants).
- `src/mjlab_microduck/tasks/backlash.py` — wraps any env cfg into its backlash twin.
- `src/mjlab_microduck/robot/microduck_constants.py` — robot cfgs, HOME frame, BAM actuator cfg.
- `src/mjlab_microduck/robot/microduck/` — MJCF exports from Onshape
  ... Collision families: `walk` (feet only), `groundcontact` (curated floor set),
  `allcollisions` (every part; XL330 housings named `*_servo_collision` by
  `name_servo_collision_geoms` → VelStand's servo-impact sensor). Each has a
  `_backlash` twin generated by `add_backlash.py <xml> --backlash-deg 2.0`.

# Domain randomization toggles
ENABLE_COM_RANDOMIZATION = True
ENABLE_HEAD_COM_RANDOMIZATION = True  # Randomize CoM of the head assembly bodies
ENABLE_KP_RANDOMIZATION = False #  Was True
ENABLE_KD_RANDOMIZATION = False #  Was True
ENABLE_MASS_INERTIA_RANDOMIZATION = True  # Can enable once walking is stable
ENABLE_JOINT_FRICTION_RANDOMIZATION = True  # Scales BAM's friction budget per-env via FrictionDRBamActuator.friction_scale
ENABLE_JOINT_DAMPING_RANDOMIZATION = False
ENABLE_ARMATURE_RANDOMIZATION = True  # Reflected rotor inertia (microban-style). DOES affect BAM (armature is set, not zeroed).
ENABLE_VELOCITY_PUSHES = True  # Velocity-based pushes for robustness training
ENABLE_IMU_ORIENTATION_RANDOMIZATION = True  # Simulates mounting errors
ENABLE_ENCODER_BIAS = True  # Per-env joint encoder calibration offset (actor obs sees joint_pos + bias)
ENABLE_BASE_ORIENTATION_RANDOMIZATION = False  # Randomize initial tilt to force reactive behavior

Actor observation layout (61-dim flat tensor, concatenated in term insertion order):
    [0:3]   base_ang_vel      (roll, pitch, yaw  — body-frame IMU)
    [3:6]   projected_gravity (gx, gy, gz         — body-frame)
    [6:20]  joint_pos_rel     (14 joints, relative to default pose)
    [20:34] joint_vel_rel     (14 joints)
    [34:48] last_action       (14 joints)
    [48:51] twist command     (lin_vel_x, lin_vel_y, ang_vel_z)
    [51:55] head command      (neck_pitch, head_pitch, head_yaw, head_roll deltas)
    [55:61] body command      (x, y, z, roll, pitch, yaw deltas)

Joint ordering within each 14-dim block (from robot_walk.xml body tree):
    0: left_hip_yaw    5: neck_pitch    9:  right_hip_yaw
    1: left_hip_roll   6: head_pitch    10: right_hip_roll
    2: left_hip_pitch  7: head_yaw      11: right_hip_pitch
    3: left_knee       8: head_roll     12: right_knee
    4: left_ankle                       13: right_ankle

- **Obs layout is 61D (actor) and shared across the whole policy family** so
  policies are hot-swappable in the runtime: 48 base proprioception +
  13D command block `[twist(3), head_pose(4), body_pose(6)]`, in that order.
  An env that doesn't use a command slot ZERO-PADS it (keep the obs term,
  sample tiny ranges) — never delete a slot.

    # Pose reward operates on LEG joints only. Head/neck are command-driven
    # (head_pose_tracking) — if they were in this reward too, it would pull
    # them to HOME while head_pose_tracking pulls them to the command, and the
    # policy converges to "ignore the command" because pose reward dominates
    # once head_pose_tracking's gradient dies at large commands.
    cfg.rewards["pose"].params["asset_cfg"] = SceneEntityCfg(
        "robot", joint_names=(r"^(?!passive_|.*neck.*|.*head.*).*",)
    )
    cfg.rewards["pose"].weight = 1.0

    # upright: deliberately strong (2.0 / std²=0.05, was 1.0 / std²=0.1).
    # 2026-07 pitch-vs-speed eval: the policy walks with a +2-4° steady forward
    # lean (p90 ~6-8°) and ~2/3 of push-induced falls at speed are FORWARD. At
    # weight 1.0 / std²=0.1 a 4° lean cost ~0.05/step — effectively free. At
    # 2.0 / std²=0.05 it costs ~0.19/step: enough gradient to hold the trunk
    # level in steady gait while transient lean (push recovery, accel) stays
    # affordable.
    cfg.rewards["upright"].weight = 2.0
    cfg.rewards["upright"].params["std"] = math.sqrt(0.05)

    # foot_slip deliberately weak (-0.1, not -1.0): -1.0 was too restrictive
    # for this robot's pivot-heavy turning.
    cfg.rewards["foot_slip"].weight = -0.1

# Fraction of envs commanded to spin on the spot (lin=0, |ang| ∈ [0.4·max, max]).
TURN_IN_PLACE_FRACTION = 0.15

    # Head droop fix (2026-08-20). The head walks pitched ~15° down (measured:
    # run ww1g2198 head_pose_tracking 1.544/2.0 → 14.6° mean joint error).
    # DO NOT fix this by tightening head_pose_tracking's std: run 5yay13u4 tried
    # fine_std=0.1 and the policy stopped walking entirely by iter 300 (air_time
    # 1.01 → 0.02, peak foot height 15 mm → 2 mm, entropy collapsed 10.9 → 1.9).
    # An instantaneous tight tolerance taxes walking 0.77/step — 76% of the whole
    # air_time reward — and is UNESCAPABLE, since a 280 g head (38% of robot
    # mass) must oscillate while stepping. Standing still scored higher, so it
    # stood still.
    # The DC bias, unlike the oscillation, IS escapable (bias the neck command up
    # to cancel gravity sag), so price only that: L1 on a 1 s EMA of the error.
    # At the optimum this costs a walking policy nothing.
    cfg.rewards["head_pose_bias"] = RewardTermCfg(
        func=microduck_mdp.head_pose_bias_penalty,
        weight=0.0,  # ramped by the head_pose_bias_weight curriculum below
        params={"command_name": "_head_pose", "tau_s": 1.0},
    )

"""BAM actuator with per-env friction-magnitude domain randomization.

The canonical ``bam.mjlab.BamActuator`` exposes per-env gain scaling (kp/kd) but
no friction hook, and under BAM MuJoCo's ``dof_frictionloss`` is zeroed in
``edit_spec`` (BAM computes friction itself in ``compute()``). So the stock
``dr.dof_frictionloss`` is a no-op here.

This thin subclass adds a per-env ``friction_scale`` that multiplies BAM's
velocity-INDEPENDENT friction budget (Coulomb + Stribeck + load-dependent) inside
``_compute_friction_budget`` — the term that carries the dominant sim2real
friction uncertainty (stiction / gearbox). The viscous (velocity-proportional)
term is left at nominal; scale it too by overriding ``compute`` if ever needed.

Non-accumulating: ``friction_scale`` is reset to 1.0 then set to a fresh sample
each episode by the ``randomize_bam_friction`` event (see tasks/mdp.py).
"""

class BacklashEncoderBamActuator(FrictionDRBamActuator):
    """FrictionDRBamActuator whose firmware PD reads the encoder THROUGH backlash.

    Backlash models (robot_groundcontact_backlash.xml) put an unactuated
    ``passive_<joint>_backlash`` hinge in series with each servo joint: the
    servo joint is the motor output, the backlash joint is the play between it
    and the link, and the link angle is their sum.

    On the real servo the magnetic encoder sits on the OUTPUT side of that
    play, so the firmware position loop closes on main+backlash — while the
    servo winds through the dead zone the measured position (and hence the PD
    error) doesn't change. This subclass reproduces that: ``cmd.pos`` fed to
    BAM's voltage control law becomes qpos[main] + qpos[backlash].

    ``cmd.vel`` is left motor-side on purpose: in BAM it drives back-EMF and
    friction, which are rotor physics, not an encoder-derived firmware signal.

    Degrades to a plain FrictionDRBamActuator on models without backlash
    joints (per-joint mask), so it is safe to use on any microduck model.
    """

    def get_command(self, data) -> ActuatorCmd:
        cmd = super().get_command(data)
        pos = cmd.pos + data.joint_pos[:, self._backlash_joint_ids] * self._backlash_mask
        return dataclasses.replace(cmd, pos=pos)

"""Inject gearbox-backlash joints into an onshape-to-robot MJCF export.

For every actuated servo joint (``class="chosen_actuator"``) this inserts an
unactuated hinge on the same body / same axis right after it:

    <joint axis="0 0 1" name="left_hip_yaw" ... class="chosen_actuator"/>
    <joint axis="0 0 1" name="passive_left_hip_yaw_backlash" class="backlash"/>

The composite link rotation is main + backlash: the main joint is the servo
output (BAM drives it), the backlash joint is the play between the servo and
the link, free to wander within ±(backlash/2).

Naming: the ``passive_`` prefix means the new joints are automatically excluded
by every existing regex in the task configs (actuators ``^(?!passive_).*``,
joint obs, pose reward).

``--backlash-deg`` is the TOTAL peak-to-peak play (what you measure wiggling
the horn with the servo held); the joint range is symmetric ±deg/2.
"""

- **Obs normalization is ON** → the normalizer must be baked into the ONNX.
  `scripts/export.py` does this; in-sim play hides the bug (it applies the
  normalizer anyway), so never hand-convert a checkpoint.
- **Policies are UNFILTERED** (no action low-pass in training). Don't add EMA
  filtering without a matched runtime flag and a transfer test — trained-with /
  deployed-without (either direction) breaks transfer.
- **Domain randomization must not accumulate across resets.** mjlab 1.3.0's
  `dr.*` ops with `operation="add"/"scale"` are natively non-accumulating (they
  re-read compile-time defaults); custom DR functions must restore-then-apply.
  An accumulating CoM randomizer once degraded every long run for months.
- If an obs is remapped to a sensor view (backlash encoder, bias), any tracking
  REWARD on the same quantity must measure the same view — otherwise the policy
  is punished for correcting what it sees.

2. **Verify physics assumptions in sim BEFORE training** — this is the single
   biggest time-saver:
   - A target/rest pose must be a stable equilibrium: hold its ctrl for 3 s
     from noisy inits and check TILT, not just height (a settle test that only
     records z reports fallen states as "resting fine").
   - Measure target heights off the actual robot in sim (e.g. trunk z under a
     standing policy), never carry them across model revisions. A 5 mm-wrong
     STAND_Z once turned the goal into an impossible target for days.

GPU 训练（mjlab，BAM 执行器）
  → play 回放（wandb run 复查指标：air_time、熵、追踪得分）
  → CPU infer_policy.py（独立运行时彩排，BAM M6 与训练一致）
  → duck-sim 软件在环（真守护进程 + MuJoCo 身体，验证部署链路）
  → 真机（保守指令起步）

"""Export a trained checkpoint to ONNX, with the observation normalizer baked in.

This is the ONE path from a checkpoint to a deployable `.onnx`: `runner.export_policy_to_onnx`
emits `actor(normalizer(obs))`, so what the robot runs is what training saw. In-sim `play`
applies the normalizer itself and hides a hand-converted checkpoint that forgot it — never
convert by hand.

`scripts/export.py` is the command-line wrapper; `mjlab_microduck.publish` calls
:func:`run_export` directly so a published policy cannot skip this step.
"""

#  ... 61 = 48 proprioception + 13 command; 14 = the servos.
OBS_LEN = 61
ACTION_LEN = 14

def check_onnx(path: Path) -> OnnxShape:
    """Refuse a file the daemon would refuse at load: wrong widths, or one that is not 61 -> 14."""
    ...
    if shape.obs_len != OBS_LEN:
        raise ManifestError(
            f"{path.name}: observation width is {shape.obs_len}, the robot builds {OBS_LEN} "
        )
    if shape.action_len != ACTION_LEN:
        raise ManifestError(f"{path.name}: {shape.action_len} actions, the robot has {ACTION_LEN}")
    return shape

uv run publish --task <TASK_ID> --wandb-run-path <...> --checkpoint N \
  --repo <user>/microduck-<name> --kind episodic --duration-s 4.0

    // ── policy.* ─────────────────────────────────────────────────────────────
    /// What is installed and what the Hub offers; see [`method::POLICY_CHECK`].
    PolicyCheck,
    /// Install a set and make it live; see [`method::POLICY_INSTALL`].
    PolicyInstall(PolicyInstallParams),
    /// Fetch one policy into the library; see [`method::POLICY_FETCH`].
    PolicyFetch(PolicyFetchParams),
    /// Search the Hub; see [`method::POLICY_SEARCH`].
    PolicySearch(PolicySearchParams),

# 0. 冒烟（纪律，不是可选）
uv run train <TASK_ID> --env.scene.num-envs 64 --agent.max_iterations 5

# 1. 正式训练（无本地 GPU 加 --hf-jobs）
uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096

# 2. 回放复查（看 air_time / 熵 / 追踪得分，记下 run id）
uv run play <TASK_ID> --wandb-run-path <entity/project/run_id>

# 3. 导出（唯一合法路径，归一化器进图）
uv run scripts/export.py <TASK_ID> --wandb-run-path <...>

# 4. CPU 部署彩排（BAM M6 执行器与训练一致）
uv run scripts/infer_policy.py --walking output.onnx

# 5. 软件在环（真守护进程 + MuJoCo 身体）
cd ../microduck && scripts/duck-sim ctl "policy fetch <user>/microduck-walk"

# 6. 发布（上传侧先跑形状门禁与来源记录）
uv run publish --task <TASK_ID> --wandb-run-path <...> \
  --repo <user>/microduck-<name> --kind episodic --duration-s 4.0

# 7. 真机（有鸭子的话）
robotctl policy add <user>/microduck-<name>

第 1–2 周   Rust 基础；通读主仓库 README、docs/design/architecture.md、robotd-design.md
第 3 周     scripts/duck-sim 起鸭；robotctl 遍历命令；对照第 3 章精读 duck-ipc-proto 的 Call 枚举
第 4–5 周   精读 robotd/src/control.rs（750 行，纯计算可单测）与 duck-control/src/obs.rs；
            用 scripts/duck-sim boot 进容器，观察 systemd 单元与健康门禁
第 6 周起   挑 docs/project/ 里的已知问题或 CONTRIBUTING.md 的入门议题，提第一个 PR

第 1 周   概念补课（PPO、域随机化）；跑 uv run list-envs 认识任务注册表
第 2 周   冒烟测试 → 4096 环境训 Velocity（无卡用 --hf-jobs）→ play 复查
第 3–4 周 改配方重训：动一个奖励权重（先按第 8 章"算现价"）或一个 DR 开关，
          对比 wandb 指标（air_time、熵、追踪得分）
第 5 周起 挑一个模板造新任务：episodic 特技→standup 模板；两态→sitstand；
          动作→roulade（读它的 cfg 文档注释——一条五次实验的教训链）
第 6 周起 走完 export → infer_policy → publish，发一条自己的策略到 HF Hub