Aether258/pi05_bi_bread_all_step10000

pi05_bi — bread (01+02 merged), step 10000

openpi pi05_bi checkpoint for a bimanual bread-and-ketchup task with tactile inputs. Step 10000 (3.23 epochs), held-out validation loss 0.0384.

This is the lowest held-out loss of the run through step 10000. Training is still in progress towards step 12,000, so a later checkpoint may yet beat it.

Task

Single instruction for every episode (both source repos already agreed on this string, so no unification was needed at merge time):

Firstly, use the right hand to pick up the bread and put it on the table. Secondly, use the left hand to pick up the ketchup and squeeze it onto the bread.

Data

source	episodes	frames
KaiyueChen/bread_01	293	196,281
KaiyueChen/bread_02	482	244,831
merged	775	441,112

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1). Mean episode length 568 frames -- notably shorter than the sibling two_tubes (788) and task2 (~966) datasets, so an epoch here is only 3,097 steps.

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

split	episodes
train	698 (396,399 frames)
val_seen (subset of train)	77
val_unseen (held out)	77

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

config	pi05_bi
hardware	2 x A100-80GB, FSDP
batch size	128
this checkpoint	step 10000 (~3.23 epoch; 1 epoch = 3,097 steps)
planned length	12,000 steps (3.87 epochs)
lr	cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps
	(CosineDecaySchedule defaults -- pi05_bi does not override lr_schedule)
LoRA	rank 16 on the LLM, rank 32 on the action expert
vision tower	fully fine-tuned -- the freeze filter matches only .*llm.*

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

step	train	val_seen	val_unseen	gap
0	0.6875	0.6461	0.6368	-0.0093
2000	0.0524	0.0509	0.0449	-0.0060
4000	0.0464	0.0460	0.0405	-0.0055
6000	0.0439	0.0432	0.0389	-0.0043
8000	0.0416	0.0419	0.0387	-0.0032
10000	0.0400	0.0396	0.0384	-0.0012

val_unseen fell steeply through step 6000 and has been flat since. The per-2000-step improvement was 0.0044, then 0.0016, then 0.0002, then 0.0003 -- the last two are within noise, since each validation pass covers only ~2,560 frames, which at ~568 frames per episode is about 4.5 episodes per split. In practical terms steps 6000, 8000 and 10000 (0.0389 / 0.0387 / 0.0384) are interchangeable.

The quantity that is actually moving is the gap, which narrowed monotonically across all five points: -0.0060, -0.0054, -0.0044, -0.0033, -0.0012. Over the last 2,000 steps val_seen improved 5.5% (0.0419 -> 0.0396) while val_unseen moved 0.0003. That is the signature of capacity going into fitting the training episodes rather than transferring -- overfitting has begun, it simply started from a negative gap and has not yet crossed zero.

Two things distinguish this run from the sibling two_tubes_0102 and task2_all runs, both of which had clearly turned by 1.2-1.9 epochs:

1. It kept improving past 1.9 epochs. At step 6000 (1.94 epochs) val_unseen was still dropping steeply, where the other two had already plateaued. The turn here came at roughly 2.0-2.6 epochs instead.
2. The gap is negative throughout (val_unseen below val_seen). This is not evidence of unusually good generalization -- with only ~4.5 episodes sampled per split, the sign of the gap is dominated by which particular episodes landed where. Only the val_unseen absolute trend, and the direction the gap moves, carry information.

Note the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two columns are therefore not directly comparable.

Loss values are not comparable across datasets -- normalization statistics are computed per dataset, so this run's 0.0387 does not mean it is "better" than two_tubes_0102's 0.0537.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/bread_all/
    norm_stats.json            # computed over the train split only