sii-research/tau-0-wm

τ₀-World Model

This repository contains the official checkpoints for τ₀-World Model: A Unified Video-Action World Model for Robotic Manipulation.

Project Website | GitHub

τ₀-World Model (τ₀-WM) is a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone (Wan-2.2), τ₀-WM provides two complementary interfaces:

1. Video Action Model (VAM): Jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state.
2. Action-conditioned Video Simulator: Rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores.

The model is trained on approximately 27,300 hours of diverse data including real-robot teleoperation, UMI-style interaction, and egocentric human videos.

Released Checkpoints

This repository contains two complementary checkpoints of τ₀-WM:

Component	Repository Path	Description
Video Action Model (VAM)	Repository root	Jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot states.
Action-conditioned Video Simulator	simulator/	Rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores.

Download Checkpoints

Video Action Model

The Video Action Model is stored in the repository root. Download it without the simulator checkpoint using:

hf download sii-research/tau-0-wm \
  --exclude "simulator/**" \
  --local-dir checkpoints/tau-0/vam

Action-conditioned Video Simulator

The simulator is stored under the simulator/ directory. Download only the simulator checkpoint using:

hf download sii-research/tau-0-wm \
  --include "simulator/**" \
  --local-dir checkpoints/tau-0

Action Space

The model is designed for dual-arm manipulation with the following specifications:

• State Input: Absolute poses of two end-effectors (14 channels: xyz + xyzw quaternion) and gripper state (2 channels).
• Action Output: Absolute poses of end-effectors and gripper openness (16 channels total).
• Internal Optimization: During pretraining, the model is optimized to predict relative poses (20 channels: xyz and 6d-rotation for each arm).

For detailed deployment instructions and environment setup, please refer to the official GitHub repository.

Citation

@article{zhou2026tau0wm,
  title={$\tau_0$-WM: A Unified Video-Action World Model for Robotic Manipulation},
  author={Zhou, Pengfei and Shengcong Chen and Di Chen and Jiaxu Wang and Rongjun Jin and Bingwen Zhu and Yike Pan and Songen Gu and Kuanning Wang and Shufeng Nan and Xingyu Qiu and Chenhao Qiu and Pu Yang and Yunuo Cai and Jianxiong Gao and Yifan Li and Yanwei Fu and Xiangyu Yue and Zhi Chen and Jianlan Luo},
  journal={arXiv preprint arXiv:2606.01027},
  year={2026}
}