pi0.5 — dex-warehouse full fine-tune (20-DoF)

Full fine-tune of π₀.₅ (VLM backbone + SigLIP vision encoder + action expert, 4.14 B trainable params) from lerobot/pi05_base on Kasra99/dex-warehouse — teleoperated warehouse pick-and-place on a Dexmate Vega-1 Pro mobile manipulator.

Checkpoints are stored per step as step_XXXXXX/, each containing EMA weights (decay 0.99, openpi-style), which are the weights intended for inference.

from huggingface_hub import snapshot_download
from lerobot.policies.pi05.modeling_pi05 import PI05Policy
from lerobot.policies.factory import make_pre_post_processors

STEP = "step_005000"          # see "Which checkpoint" below
root = snapshot_download("Kasra99/pi05_dex_warehouse_full", allow_patterns=f"{STEP}/*")
path = f"{root}/{STEP}"

policy = PI05Policy.from_pretrained(path).eval().cuda()
pre, post = make_pre_post_processors(
    policy.config, pretrained_path=path,
    preprocessor_overrides={"device_processor": {"device": "cuda"}},
)

obs = {                                    # 20-dim state, these exact camera keys
    "observation.images.base_0_rgb":        head_img,      # (3,H,W) float [0,1]
    "observation.images.left_wrist_0_rgb":  left_img,
    "observation.images.right_wrist_0_rgb": right_img,
    "observation.state":                    state20,
    "task": "pick up the bear toy and put it in the box",   # must match a trained string
}
action = post(policy.select_action(pre(obs)))   # -> (1, 20)

from_pretrained has no subfolder argument, so download the step folder first and pass the local path, as above.

Which checkpoint

Eval loss rose monotonically at all 15 measurements (0.114 @ 2k -> 0.338 @ 30k) while train loss fell to 0.015, so later checkpoints are progressively more memorized. Start with step_005000, then step_010000. See Limitations.

Action space (20-DoF)

arm_center_z, L_arm_j1..j7, R_arm_j1..j7, right_hand.{open_close,thumb_opposition}_ratio, base_{vx,vy,wz}.

The two left-hand dims present in the raw dataset were dropped: the left hand is never actuated (exactly 0.0 in 217,846 / 219,260 frames), and its q01–q99 span of ~0.01 made pi0.5's QUANTILES normalization map the rare closures to +199, injecting targets ~200x normal scale into the flow-matching loss. After removal the worst normalized action is ~10.

Cameras are named for the slots pi05_base was pretrained with: observation.images.base_0_rgb (head), .left_wrist_0_rgb, .right_wrist_0_rgb.

Training


base	`lerobot/pi05_base` (lerobot 0.6.2)
trainable	full — backbone + vision encoder + action expert
precision	fp32 master weights, bf16 autocast
batch / steps	32 / 30,000 (≈4.4 epochs)
lr	2.5e-5 peak, cosine → 2.5e-6, 1k warmup
chunk size	50 (1.67 s @ 30 fps)
EMA	0.99 constant
hardware	1x RTX PRO 6000 Blackwell, ~32 h

Limitations

This is right-arm manipulation on a mobile base, not bimanual manipulation. Measured over all 177 episodes: the left arm has 1.75x the path length of the right (81.0 vs 50.7 rad/episode) but 2.6x lower cross-episode reproducibility (R² 0.087 vs 0.226) and higher jitter, and its hand never actuates — the signature of an idle teleoperator controller rather than task work. The left-arm joints are retained in the action space but should not be expected to produce purposeful motion.

The base is the most reproducible channel (R² 0.274) and is commanded in ~34% of frames.

Two of the 22 task strings have only one episode each and were held out entirely by the 5% per-task eval split, so they have no training data.