Spaces:
Running on Zero
Running on Zero
Deploy ARDY ZeroGPU Blender motion API
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +1 -0
- README.md +45 -6
- app.py +1113 -0
- ardy/__init__.py +4 -0
- ardy/assets.py +19 -0
- ardy/assets/skeletons/cskel27/joints.p +0 -0
- ardy/assets/skeletons/cskel27/skin_standard.npz +3 -0
- ardy/assets/skeletons/cskel29/joints.p +0 -0
- ardy/assets/skeletons/cskel29/standard_t_pose_cskel29_joint_quat.p +0 -0
- ardy/assets/skeletons/g1skel34/joints.p +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/head_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_ankle_pitch_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_ankle_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_elbow_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_pitch_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_knee_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_rubber_hand.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_pitch_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_pitch_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/logo_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/pelvis.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/pelvis_contour_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_ankle_pitch_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_ankle_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_elbow_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_pitch_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_knee_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_rubber_hand.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_pitch_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_pitch_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_roll_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/torso_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/torso_link_rev_1_0.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/waist_roll_link.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/waist_roll_link_rev_1_0.STL +0 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/waist_yaw_link.STL +3 -0
- ardy/assets/skeletons/g1skel34/meshes/g1/waist_yaw_link_rev_1_0.STL +3 -0
- ardy/assets/skeletons/g1skel34/xml/g1.xml +413 -0
- ardy/assets/skeletons/somaskel30/joints.p +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
*.STL filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,13 +1,52 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.20.0
|
| 8 |
-
python_version: '3.13'
|
| 9 |
app_file: app.py
|
|
|
|
|
|
|
|
|
|
| 10 |
pinned: false
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: ARDY Motion Playground
|
| 3 |
+
emoji: 🕺
|
| 4 |
+
colorFrom: gray
|
| 5 |
+
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.20.0
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
+
short_description: Text-to-motion with an interactive 3D skeleton playground
|
| 10 |
+
python_version: "3.12"
|
| 11 |
+
startup_duration_timeout: 1h
|
| 12 |
pinned: false
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# ARDY Motion Playground
|
| 16 |
+
|
| 17 |
+
Interactive **text-to-motion** demo for [ARDY](https://research.nvidia.com/labs/sil/projects/ardy/)
|
| 18 |
+
(*Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation*, NVIDIA, SIGGRAPH 2026).
|
| 19 |
+
|
| 20 |
+
Pick a **rig** (human or robot), type a natural-language prompt, generate a 3D motion clip,
|
| 21 |
+
then **orbit, scrub, and play** it in the built-in Three.js playground (play/pause, timeline
|
| 22 |
+
scrubber, speed control, loop, and a root-trajectory trail).
|
| 23 |
+
|
| 24 |
+
- **Human rig:** `nvidia/ARDY-Core-RP-20FPS-Horizon40` — 27-joint "Core" skeleton at 20 fps.
|
| 25 |
+
- **Robot rig:** `nvidia/ARDY-G1-RP-25FPS-Horizon52` — 34-joint Unitree G1 humanoid robot at 25 fps
|
| 26 |
+
(same rig as the sibling Space `hugging-apps/ardy-g1-motion-generation`).
|
| 27 |
+
- **Text encoder:** LLM2Vec-Llama-3-8B using the public
|
| 28 |
+
`unsloth/llama-3-8b-Instruct` base plus the original McGill MNTP and supervised
|
| 29 |
+
adapters. No Meta-gated base-model access is required.
|
| 30 |
+
- **Hardware:** ZeroGPU.
|
| 31 |
+
|
| 32 |
+
Post-processing (foot-skate cleanup, which needs a compiled C++ extension) is disabled in this
|
| 33 |
+
demo; generation uses the model's autoregressive path directly.
|
| 34 |
+
|
| 35 |
+
Based on the [nv-tlabs/ardy](https://github.com/nv-tlabs/ardy) reference implementation
|
| 36 |
+
(`scripts/generate.py`).
|
| 37 |
+
|
| 38 |
+
## Blender export
|
| 39 |
+
|
| 40 |
+
Open **Blender export** to generate three downloadable files:
|
| 41 |
+
|
| 42 |
+
- `ardy_motion.bvh` — directly importable with Blender's BVH importer.
|
| 43 |
+
- `ardy_motion.npz` — complete ARDY output including joint positions, local and
|
| 44 |
+
global rotations, root trajectory, and foot contacts.
|
| 45 |
+
- `metadata.json` — prompt, seed, FPS, skeleton hierarchy, and coordinate-system
|
| 46 |
+
information.
|
| 47 |
+
|
| 48 |
+
The programmatic endpoint is `/generate_blender`. The existing `/generate`
|
| 49 |
+
endpoint and Three.js playground remain available.
|
| 50 |
+
|
| 51 |
+
The base and adapter repositories can be overridden with the Space variables
|
| 52 |
+
`LLM2VEC_BASE_MODEL`, `LLM2VEC_MNTP_MODEL`, and `LLM2VEC_SUPERVISED_MODEL`.
|
app.py
ADDED
|
@@ -0,0 +1,1113 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os
|
| 2 |
+
|
| 3 |
+
os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
|
| 4 |
+
# Load models locally (no external text-encoder service) and prefer the HF cache.
|
| 5 |
+
os.environ.setdefault("TEXT_ENCODER_MODE", "local")
|
| 6 |
+
|
| 7 |
+
import base64
|
| 8 |
+
import gzip
|
| 9 |
+
import json
|
| 10 |
+
import random
|
| 11 |
+
import sys
|
| 12 |
+
import tempfile
|
| 13 |
+
import time
|
| 14 |
+
import xml.etree.ElementTree as ET
|
| 15 |
+
from pathlib import Path
|
| 16 |
+
|
| 17 |
+
import spaces # must precede torch / CUDA-touching imports
|
| 18 |
+
import torch
|
| 19 |
+
import numpy as np
|
| 20 |
+
import trimesh
|
| 21 |
+
import gradio as gr
|
| 22 |
+
|
| 23 |
+
# The vendored ARDY package lives next to this file.
|
| 24 |
+
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
| 25 |
+
|
| 26 |
+
from ardy.model import load_model # noqa: E402
|
| 27 |
+
from ardy.exports import write_bvh # noqa: E402
|
| 28 |
+
from ardy.model.load_model import load_text_encoder # noqa: E402
|
| 29 |
+
from ardy.motion_rep.tools import length_to_mask # noqa: E402
|
| 30 |
+
from ardy.tools import seed_everything, to_numpy # noqa: E402
|
| 31 |
+
|
| 32 |
+
# Two rigs are offered in the playground:
|
| 33 |
+
# "human" -> ARDY-Core-RP-20FPS-Horizon40, 27-joint skeleton @ 20 fps
|
| 34 |
+
# "robot" -> ARDY-G1-RP-25FPS-Horizon52, 34-joint Unitree G1 robot @ 25 fps
|
| 35 |
+
# The robot rig matches the sibling Space hugging-apps/ardy-g1-motion-generation.
|
| 36 |
+
DEFAULT_RIG = "human"
|
| 37 |
+
MAX_SEED = 2**31 - 1
|
| 38 |
+
|
| 39 |
+
# -----------------------------------------------------------------------------
|
| 40 |
+
# Model loading (module scope).
|
| 41 |
+
#
|
| 42 |
+
# ZeroGPU has no live GPU at startup: it intercepts .to("cuda") *placement* but
|
| 43 |
+
# NOT arbitrary CUDA compute. LLM2Vec's PEFT load runs a LoRA `merge_and_unload`
|
| 44 |
+
# (real matmuls), so we build everything on CPU first, then .to("cuda") — the
|
| 45 |
+
# placement call is what the ZeroGPU hijack packs to disk and streams into VRAM
|
| 46 |
+
# on the first @spaces.GPU request.
|
| 47 |
+
# -----------------------------------------------------------------------------
|
| 48 |
+
# transformers / PEFT / safetensors infer their load device from
|
| 49 |
+
# torch.cuda.is_available(), which ZeroGPU reports True at startup even though no
|
| 50 |
+
# real GPU is attached — so a plain load tries to place weights on cuda and dies
|
| 51 |
+
# with "No CUDA GPUs are available". Force every loader onto CPU by masking
|
| 52 |
+
# is_available() during construction, then restore it and .to("cuda") (which the
|
| 53 |
+
# ZeroGPU hijack packs to disk + streams into VRAM on the first request).
|
| 54 |
+
#
|
| 55 |
+
# A single text encoder is built once and shared across both motion models
|
| 56 |
+
# (load_text_encoder is explicitly designed for this — see its docstring).
|
| 57 |
+
_real_cuda_available = torch.cuda.is_available
|
| 58 |
+
torch.cuda.is_available = lambda: False
|
| 59 |
+
try:
|
| 60 |
+
print("Loading ARDY text encoder (LLM2Vec-Llama-3-8B) on CPU…", flush=True)
|
| 61 |
+
_text_encoder = load_text_encoder(mode="local", device="cpu")
|
| 62 |
+
|
| 63 |
+
print("Loading ARDY human (core) motion model on CPU…", flush=True)
|
| 64 |
+
MODEL_HUMAN = load_model("core", device="cpu", text_encoder=_text_encoder)
|
| 65 |
+
|
| 66 |
+
print("Loading ARDY robot (G1) motion model on CPU…", flush=True)
|
| 67 |
+
MODEL_ROBOT = load_model("g1", device="cpu", text_encoder=_text_encoder)
|
| 68 |
+
finally:
|
| 69 |
+
torch.cuda.is_available = _real_cuda_available
|
| 70 |
+
|
| 71 |
+
print("Moving models to CUDA (ZeroGPU-intercepted placement)…", flush=True)
|
| 72 |
+
_text_encoder = _text_encoder.to("cuda")
|
| 73 |
+
for _m in (MODEL_HUMAN, MODEL_ROBOT):
|
| 74 |
+
_m.to("cuda")
|
| 75 |
+
# self.device was captured at construction (device="cpu"); generation creates
|
| 76 |
+
# many tensors on it, so retarget it to cuda to match the moved weights.
|
| 77 |
+
_m.device = "cuda"
|
| 78 |
+
_m.eval()
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def _rig_params(model):
|
| 82 |
+
fps = float(model.motion_rep.fps)
|
| 83 |
+
skeleton = model.skeleton
|
| 84 |
+
parents = skeleton.joint_parents.cpu().numpy().astype(int).tolist()
|
| 85 |
+
patch = model.num_frames_per_token
|
| 86 |
+
gen_horizon = model.gen_horizon_len
|
| 87 |
+
num_base_steps = int(model.diffusion.num_base_steps)
|
| 88 |
+
# History carried between autoregressive windows. The reference streaming demo
|
| 89 |
+
# keeps this SHORT so a new prompt takes effect within a window.
|
| 90 |
+
hist_crop = max(patch, (4 // patch) * patch)
|
| 91 |
+
root_idx = parents.index(-1) if -1 in parents else 0
|
| 92 |
+
return {
|
| 93 |
+
"model": model,
|
| 94 |
+
"fps": fps,
|
| 95 |
+
"skeleton": skeleton,
|
| 96 |
+
"parents": parents,
|
| 97 |
+
"patch": patch,
|
| 98 |
+
"gen_horizon": gen_horizon,
|
| 99 |
+
"num_base_steps": num_base_steps,
|
| 100 |
+
"hist_crop": hist_crop,
|
| 101 |
+
"root_idx": root_idx,
|
| 102 |
+
}
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
RIGS = {
|
| 106 |
+
"human": _rig_params(MODEL_HUMAN),
|
| 107 |
+
"robot": _rig_params(MODEL_ROBOT),
|
| 108 |
+
}
|
| 109 |
+
# The largest base-step count across rigs bounds the diffusion-steps slider.
|
| 110 |
+
NUM_BASE_STEPS = max(r["num_base_steps"] for r in RIGS.values())
|
| 111 |
+
for _name, _r in RIGS.items():
|
| 112 |
+
print(
|
| 113 |
+
f"[{_name}] rig ready: {_r['skeleton'].nbjoints} joints, {_r['fps']} fps, "
|
| 114 |
+
f"horizon {_r['gen_horizon']}, patch {_r['patch']}, "
|
| 115 |
+
f"hist_crop {_r['hist_crop']}, base_steps {_r['num_base_steps']}",
|
| 116 |
+
flush=True,
|
| 117 |
+
)
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
def _normalize_rig(rig) -> str:
|
| 121 |
+
"""Map any UI/API rig value onto a valid RIGS key ('human' | 'robot')."""
|
| 122 |
+
if rig is None:
|
| 123 |
+
return DEFAULT_RIG
|
| 124 |
+
key = str(rig).strip().lower()
|
| 125 |
+
if key in RIGS:
|
| 126 |
+
return key
|
| 127 |
+
if key.startswith("hum") or "core" in key or "person" in key:
|
| 128 |
+
return "human"
|
| 129 |
+
if key.startswith("rob") or "g1" in key or "unitree" in key:
|
| 130 |
+
return "robot"
|
| 131 |
+
return DEFAULT_RIG
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
# -----------------------------------------------------------------------------
|
| 135 |
+
# HUMAN rig: skinned body mesh (ARDY "CoreSkin" linear-blend skinning).
|
| 136 |
+
#
|
| 137 |
+
# The reference viz (ardy/viz/viser_utils.py) renders a smooth humanoid body by
|
| 138 |
+
# skinning a bind mesh with the per-frame *global* joint transforms:
|
| 139 |
+
# verts = CoreSkin.skin(global_rot_mats, posed_joints, rot_is_global=True)
|
| 140 |
+
# The browser holds the static skin data (bind vertices / faces / LBS
|
| 141 |
+
# indices+weights) and does the per-vertex blend, while the server sends only the
|
| 142 |
+
# tiny per-frame joint affine matrices
|
| 143 |
+
# A[f,j] = fk[f,j] @ bind_rig_transform_inv[j] (fk = [R_global | pos])
|
| 144 |
+
# so the payload stays small (~0.1 MB/clip).
|
| 145 |
+
# -----------------------------------------------------------------------------
|
| 146 |
+
_HUMAN_SKEL = RIGS["human"]["skeleton"]
|
| 147 |
+
_SKIN_PATH = Path(_HUMAN_SKEL.folder) / "skin_standard.npz"
|
| 148 |
+
_skin = np.load(_SKIN_PATH)
|
| 149 |
+
BIND_RIG_INV = np.linalg.inv(
|
| 150 |
+
np.asarray(_skin["bind_rig_transform"], dtype=np.float64)
|
| 151 |
+
).astype(np.float32) # [J, 4, 4]
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
def _build_skin_blob():
|
| 155 |
+
"""Pack the static skin data into one gzip+base64 blob (loaded once by the
|
| 156 |
+
browser). Layout: bind_vertices f32[V,3] | faces u32[F,3] | lbs_idx u8[V,W]
|
| 157 |
+
| lbs_wt f32[V,W]."""
|
| 158 |
+
bind_v = np.asarray(_skin["bind_vertices"], dtype=np.float32)
|
| 159 |
+
faces = np.asarray(_skin["faces"], dtype=np.uint32)
|
| 160 |
+
idx = np.asarray(_skin["lbs_indices"], dtype=np.uint8)
|
| 161 |
+
wt = np.asarray(_skin["lbs_weights"], dtype=np.float32)
|
| 162 |
+
raw = (
|
| 163 |
+
np.ascontiguousarray(bind_v).tobytes()
|
| 164 |
+
+ np.ascontiguousarray(faces).tobytes()
|
| 165 |
+
+ np.ascontiguousarray(idx).tobytes()
|
| 166 |
+
+ np.ascontiguousarray(wt).tobytes()
|
| 167 |
+
)
|
| 168 |
+
meta = {"V": int(bind_v.shape[0]), "F": int(faces.shape[0]), "W": int(idx.shape[1])}
|
| 169 |
+
return base64.b64encode(gzip.compress(raw, 6)).decode("ascii"), meta
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
SKIN_B64, SKIN_META = _build_skin_blob()
|
| 173 |
+
print(f"CoreSkin ready: {SKIN_META['V']} verts / {SKIN_META['F']} faces, "
|
| 174 |
+
f"blob {len(SKIN_B64) // 1024} KB", flush=True)
|
| 175 |
+
|
| 176 |
+
|
| 177 |
+
def _joint_affines_human(global_rot_mats: np.ndarray, posed_joints: np.ndarray) -> str:
|
| 178 |
+
"""Per-frame joint affine matrices A = fk @ bind_rig_inv, base64 f32 [T,J,12].
|
| 179 |
+
|
| 180 |
+
global_rot_mats: [T, J, 3, 3]; posed_joints: [T, J, 3]."""
|
| 181 |
+
T, J = posed_joints.shape[:2]
|
| 182 |
+
fk = np.tile(np.eye(4, dtype=np.float32), (T, J, 1, 1))
|
| 183 |
+
fk[..., :3, :3] = global_rot_mats.astype(np.float32)
|
| 184 |
+
fk[..., :3, 3] = posed_joints.astype(np.float32)
|
| 185 |
+
A = (fk @ BIND_RIG_INV)[..., :3, :] # [T, J, 3, 4]
|
| 186 |
+
A = np.ascontiguousarray(A.reshape(T, J, 12).astype(np.float32))
|
| 187 |
+
return base64.b64encode(A.tobytes()).decode("ascii")
|
| 188 |
+
|
| 189 |
+
|
| 190 |
+
# -----------------------------------------------------------------------------
|
| 191 |
+
# ROBOT rig: G1 robot mesh rig (rigid per-joint STL meshes).
|
| 192 |
+
#
|
| 193 |
+
# Unlike the human skeleton (rendered with one skinned body mesh via LBS), the
|
| 194 |
+
# Unitree G1 robot is rendered by attaching a rigid STL mesh to each articulated
|
| 195 |
+
# joint. This mirrors ardy/viz/g1_rig.py (G1MeshRig): each mesh has a local
|
| 196 |
+
# transform (geom_pos, geom_rot) relative to its joint, read from the MuJoCo
|
| 197 |
+
# g1.xml, plus a coordinate change from MuJoCo to ARDY axes. We precompute — for
|
| 198 |
+
# each mesh — its geometry PRE-TRANSFORMED into the joint-local frame
|
| 199 |
+
# (v' = geom_rot @ v + geom_pos), so at render time the browser just applies the
|
| 200 |
+
# per-frame joint transform: world_v = joint_pos + joint_rot @ v'.
|
| 201 |
+
# -----------------------------------------------------------------------------
|
| 202 |
+
# G1 joint -> STL mesh mapping (mirrors ardy/viz/g1_rig.py G1_MESH_JOINT_MAP).
|
| 203 |
+
G1_MESH_JOINT_MAP = {
|
| 204 |
+
"pelvis_skel": ["pelvis.STL", "pelvis_contour_link.STL"],
|
| 205 |
+
"left_hip_pitch_skel": ["left_hip_pitch_link.STL"],
|
| 206 |
+
"left_hip_roll_skel": ["left_hip_roll_link.STL"],
|
| 207 |
+
"left_hip_yaw_skel": ["left_hip_yaw_link.STL"],
|
| 208 |
+
"left_knee_skel": ["left_knee_link.STL"],
|
| 209 |
+
"left_ankle_pitch_skel": ["left_ankle_pitch_link.STL"],
|
| 210 |
+
"left_ankle_roll_skel": ["left_ankle_roll_link.STL"],
|
| 211 |
+
"right_hip_pitch_skel": ["right_hip_pitch_link.STL"],
|
| 212 |
+
"right_hip_roll_skel": ["right_hip_roll_link.STL"],
|
| 213 |
+
"right_hip_yaw_skel": ["right_hip_yaw_link.STL"],
|
| 214 |
+
"right_knee_skel": ["right_knee_link.STL"],
|
| 215 |
+
"right_ankle_pitch_skel": ["right_ankle_pitch_link.STL"],
|
| 216 |
+
"right_ankle_roll_skel": ["right_ankle_roll_link.STL"],
|
| 217 |
+
"waist_yaw_skel": ["waist_yaw_link_rev_1_0.STL", "waist_yaw_link.STL"],
|
| 218 |
+
"waist_roll_skel": ["waist_roll_link_rev_1_0.STL", "waist_roll_link.STL"],
|
| 219 |
+
"waist_pitch_skel": [
|
| 220 |
+
"torso_link_rev_1_0.STL",
|
| 221 |
+
"torso_link.STL",
|
| 222 |
+
"logo_link.STL",
|
| 223 |
+
"head_link.STL",
|
| 224 |
+
],
|
| 225 |
+
"left_shoulder_pitch_skel": ["left_shoulder_pitch_link.STL"],
|
| 226 |
+
"left_shoulder_roll_skel": ["left_shoulder_roll_link.STL"],
|
| 227 |
+
"left_shoulder_yaw_skel": ["left_shoulder_yaw_link.STL"],
|
| 228 |
+
"left_elbow_skel": ["left_elbow_link.STL"],
|
| 229 |
+
"left_wrist_roll_skel": ["left_wrist_roll_link.STL"],
|
| 230 |
+
"left_wrist_pitch_skel": ["left_wrist_pitch_link.STL"],
|
| 231 |
+
"left_wrist_yaw_skel": ["left_wrist_yaw_link.STL", "left_rubber_hand.STL"],
|
| 232 |
+
"right_shoulder_pitch_skel": ["right_shoulder_pitch_link.STL"],
|
| 233 |
+
"right_shoulder_roll_skel": ["right_shoulder_roll_link.STL"],
|
| 234 |
+
"right_shoulder_yaw_skel": ["right_shoulder_yaw_link.STL"],
|
| 235 |
+
"right_elbow_skel": ["right_elbow_link.STL"],
|
| 236 |
+
"right_wrist_roll_skel": ["right_wrist_roll_link.STL"],
|
| 237 |
+
"right_wrist_pitch_skel": ["right_wrist_pitch_link.STL"],
|
| 238 |
+
"right_wrist_yaw_skel": ["right_wrist_yaw_link.STL", "right_rubber_hand.STL"],
|
| 239 |
+
}
|
| 240 |
+
|
| 241 |
+
_ROBOT_SKEL = RIGS["robot"]["skeleton"]
|
| 242 |
+
_MUJOCO_TO_ARDY = np.array(
|
| 243 |
+
[[0.0, 1.0, 0.0], [0.0, 0.0, 1.0], [1.0, 0.0, 0.0]], dtype=np.float64
|
| 244 |
+
)
|
| 245 |
+
_G1_SKEL_DIR = Path(_ROBOT_SKEL.folder)
|
| 246 |
+
_G1_MESH_DIR = _G1_SKEL_DIR / "meshes" / "g1"
|
| 247 |
+
_G1_XML = _G1_SKEL_DIR / "xml" / "g1.xml"
|
| 248 |
+
|
| 249 |
+
|
| 250 |
+
def _quat_wxyz_to_matrix(wxyz: np.ndarray) -> np.ndarray:
|
| 251 |
+
w, x, y, z = wxyz
|
| 252 |
+
n = np.sqrt(w * w + x * x + y * y + z * z)
|
| 253 |
+
if n < 1e-12:
|
| 254 |
+
return np.eye(3)
|
| 255 |
+
w, x, y, z = w / n, x / n, y / n, z / n
|
| 256 |
+
return np.array(
|
| 257 |
+
[
|
| 258 |
+
[1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w)],
|
| 259 |
+
[2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w)],
|
| 260 |
+
[2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y)],
|
| 261 |
+
],
|
| 262 |
+
dtype=np.float64,
|
| 263 |
+
)
|
| 264 |
+
|
| 265 |
+
|
| 266 |
+
def _mesh_local_transforms() -> dict:
|
| 267 |
+
"""mesh_file -> (geom_pos[3], geom_rot[3x3]) in ARDY axes, parsed from g1.xml."""
|
| 268 |
+
if not _G1_XML.exists():
|
| 269 |
+
return {}
|
| 270 |
+
root = ET.parse(_G1_XML).getroot()
|
| 271 |
+
file_to_name = {}
|
| 272 |
+
for mesh in root.findall(".//asset/mesh"):
|
| 273 |
+
name, file = mesh.get("name"), mesh.get("file")
|
| 274 |
+
if name and file:
|
| 275 |
+
file_to_name[file] = name
|
| 276 |
+
name_to_tf = {}
|
| 277 |
+
for geom in root.findall(".//geom"):
|
| 278 |
+
name = geom.get("mesh")
|
| 279 |
+
if name is None:
|
| 280 |
+
continue
|
| 281 |
+
pos = geom.get("pos")
|
| 282 |
+
quat = geom.get("quat")
|
| 283 |
+
gp = np.zeros(3) if pos is None else np.array([float(v) for v in pos.split()])
|
| 284 |
+
gr_ = np.eye(3) if quat is None else _quat_wxyz_to_matrix(
|
| 285 |
+
np.array([float(v) for v in quat.split()])
|
| 286 |
+
)
|
| 287 |
+
name_to_tf[name] = (gp, gr_)
|
| 288 |
+
out = {}
|
| 289 |
+
for file, name in file_to_name.items():
|
| 290 |
+
gp, gr_ = name_to_tf.get(name, (np.zeros(3), np.eye(3)))
|
| 291 |
+
gp = _MUJOCO_TO_ARDY @ gp
|
| 292 |
+
gr_ = _MUJOCO_TO_ARDY @ gr_ @ _MUJOCO_TO_ARDY.T
|
| 293 |
+
out[file] = (gp, gr_)
|
| 294 |
+
return out
|
| 295 |
+
|
| 296 |
+
|
| 297 |
+
def _build_g1_mesh_blob():
|
| 298 |
+
"""Pack all rigid G1 meshes, pre-transformed into joint-local frame, into one
|
| 299 |
+
gzip+base64 blob. Returns (b64, meta). meta.parts lists per-mesh
|
| 300 |
+
{joint, v_off, v_cnt}. Layout: all verts f32[Vtot,3] then all faces
|
| 301 |
+
u32[Ftot,3] (face indices are GLOBAL into the concatenated vertex array)."""
|
| 302 |
+
skeleton = _ROBOT_SKEL
|
| 303 |
+
local_tf = _mesh_local_transforms()
|
| 304 |
+
all_v = []
|
| 305 |
+
all_f = []
|
| 306 |
+
parts = []
|
| 307 |
+
v_cursor = 0
|
| 308 |
+
for joint_name, mesh_files in G1_MESH_JOINT_MAP.items():
|
| 309 |
+
if joint_name not in skeleton.bone_index:
|
| 310 |
+
continue
|
| 311 |
+
joint_idx = int(skeleton.bone_index[joint_name])
|
| 312 |
+
for mesh_file in mesh_files:
|
| 313 |
+
mp = _G1_MESH_DIR / mesh_file
|
| 314 |
+
if not mp.exists():
|
| 315 |
+
continue
|
| 316 |
+
mesh = trimesh.load_mesh(str(mp), process=True)
|
| 317 |
+
if isinstance(mesh, trimesh.Scene):
|
| 318 |
+
mesh = trimesh.util.concatenate(mesh.dump())
|
| 319 |
+
verts = np.asarray(mesh.vertices, dtype=np.float64) @ _MUJOCO_TO_ARDY.T
|
| 320 |
+
faces = np.asarray(mesh.faces, dtype=np.int64)
|
| 321 |
+
gp, gr_ = local_tf.get(mesh_file, (np.zeros(3), np.eye(3)))
|
| 322 |
+
# Pre-apply the mesh's joint-local transform: v' = geom_rot @ v + geom_pos.
|
| 323 |
+
verts = (verts @ gr_.T) + gp
|
| 324 |
+
vcnt = verts.shape[0]
|
| 325 |
+
parts.append({"joint": joint_idx, "v_off": v_cursor, "v_cnt": vcnt})
|
| 326 |
+
all_v.append(verts.astype(np.float32))
|
| 327 |
+
all_f.append((faces + v_cursor).astype(np.uint32)) # global vertex indices
|
| 328 |
+
v_cursor += vcnt
|
| 329 |
+
V = np.concatenate(all_v, axis=0) if all_v else np.zeros((0, 3), np.float32)
|
| 330 |
+
F = np.concatenate(all_f, axis=0) if all_f else np.zeros((0, 3), np.uint32)
|
| 331 |
+
raw = np.ascontiguousarray(V).tobytes() + np.ascontiguousarray(F).tobytes()
|
| 332 |
+
meta = {"Vtot": int(V.shape[0]), "Ftot": int(F.shape[0]), "parts": parts}
|
| 333 |
+
b64 = base64.b64encode(gzip.compress(raw, 6)).decode("ascii")
|
| 334 |
+
return b64, meta
|
| 335 |
+
|
| 336 |
+
|
| 337 |
+
G1_MESH_B64, G1_MESH_META = _build_g1_mesh_blob()
|
| 338 |
+
print(
|
| 339 |
+
f"G1 rig ready: {len(G1_MESH_META['parts'])} meshes, "
|
| 340 |
+
f"{G1_MESH_META['Vtot']} verts / {G1_MESH_META['Ftot']} faces, "
|
| 341 |
+
f"blob {len(G1_MESH_B64) // 1024} KB",
|
| 342 |
+
flush=True,
|
| 343 |
+
)
|
| 344 |
+
|
| 345 |
+
|
| 346 |
+
def _joint_transforms_robot(global_rot_mats: np.ndarray, posed_joints: np.ndarray) -> str:
|
| 347 |
+
"""Per-frame joint affine matrices [R_global | pos], base64 f32 [T,J,12].
|
| 348 |
+
|
| 349 |
+
The browser applies world_v = pos + R_global @ v' per mesh (v' already in
|
| 350 |
+
joint-local frame)."""
|
| 351 |
+
T, J = posed_joints.shape[:2]
|
| 352 |
+
A = np.zeros((T, J, 3, 4), dtype=np.float32)
|
| 353 |
+
A[..., :3, :3] = global_rot_mats.astype(np.float32)
|
| 354 |
+
A[..., :3, 3] = posed_joints.astype(np.float32)
|
| 355 |
+
A = np.ascontiguousarray(A.reshape(T, J, 12).astype(np.float32))
|
| 356 |
+
return base64.b64encode(A.tobytes()).decode("ascii")
|
| 357 |
+
|
| 358 |
+
|
| 359 |
+
# --- Autoregressive generation with a persistable latent state ---------------
|
| 360 |
+
# ARDY is autoregressive: it generates one `gen_horizon_len`-frame window at a
|
| 361 |
+
# time, conditioned on a history of previous frames. `autoregressive_step` is
|
| 362 |
+
# the streaming primitive — it returns the *normalized motion-feature tensor*
|
| 363 |
+
# for (history + new window), which can be fed straight back in as the next
|
| 364 |
+
# window's history. We thread that tensor to (a) fill a requested clip length
|
| 365 |
+
# and (b) CONTINUE a clip with a new prompt, exactly like the reference
|
| 366 |
+
# interactive demo. Persisting the tensor in a gr.State lets a second "Continue"
|
| 367 |
+
# click resume from where the first clip ended, on the same character.
|
| 368 |
+
def _generate_sequence(rig_key, prompt, num_new_frames, steps, cfg_weight, init_tensor):
|
| 369 |
+
"""Run the AR loop for one prompt on the selected rig. `init_tensor`:
|
| 370 |
+
normalized feature tensor [1, Th, D] on cuda (prior motion to continue), or
|
| 371 |
+
None to start fresh. Returns the full normalized feature tensor."""
|
| 372 |
+
r = RIGS[rig_key]
|
| 373 |
+
model = r["model"]
|
| 374 |
+
gen_horizon = r["gen_horizon"]
|
| 375 |
+
patch = r["patch"]
|
| 376 |
+
hist_crop = r["hist_crop"]
|
| 377 |
+
text_feat, text_pad_mask = model._encode_text([prompt])
|
| 378 |
+
motion_tensor = init_tensor
|
| 379 |
+
target_new = max(1, int(np.ceil(num_new_frames / gen_horizon))) * gen_horizon
|
| 380 |
+
produced = 0
|
| 381 |
+
while produced < target_new:
|
| 382 |
+
if motion_tensor is None:
|
| 383 |
+
history, hist_len = None, 0
|
| 384 |
+
else:
|
| 385 |
+
hist_len = (min(motion_tensor.shape[1], hist_crop) // patch) * patch
|
| 386 |
+
history = motion_tensor[:, motion_tensor.shape[1] - hist_len:] if hist_len else None
|
| 387 |
+
hist_len = history.shape[1] if history is not None else 0
|
| 388 |
+
samples = model.autoregressive_step(
|
| 389 |
+
num_frames=hist_len + gen_horizon, # exactly history + one window (no future)
|
| 390 |
+
num_denoising_steps=steps,
|
| 391 |
+
motion_mask=None,
|
| 392 |
+
observed_motion=None,
|
| 393 |
+
cfg_weight=float(cfg_weight),
|
| 394 |
+
texts=None,
|
| 395 |
+
text_feat=text_feat,
|
| 396 |
+
text_pad_mask=text_pad_mask,
|
| 397 |
+
init_history_sequence=history,
|
| 398 |
+
init_global_translation=None, # first window -> defaults (origin / +Z heading)
|
| 399 |
+
init_first_heading_angle=None,
|
| 400 |
+
)
|
| 401 |
+
new_tail = samples[:, hist_len:] # the freshly generated window
|
| 402 |
+
motion_tensor = new_tail if motion_tensor is None else torch.cat([motion_tensor, new_tail], dim=1)
|
| 403 |
+
produced += new_tail.shape[1]
|
| 404 |
+
return motion_tensor
|
| 405 |
+
|
| 406 |
+
|
| 407 |
+
def _decode_motion(rig_key, motion_tensor):
|
| 408 |
+
r = RIGS[rig_key]
|
| 409 |
+
model = r["model"]
|
| 410 |
+
with torch.no_grad():
|
| 411 |
+
out = to_numpy(model.motion_rep.inverse(motion_tensor, is_normalized=True))
|
| 412 |
+
return {
|
| 413 |
+
key: np.asarray(value)[0]
|
| 414 |
+
for key, value in out.items()
|
| 415 |
+
}
|
| 416 |
+
|
| 417 |
+
|
| 418 |
+
def _pack_payload(rig_key, motion_tensor, prompt, seed):
|
| 419 |
+
"""Decode a normalized feature tensor to the browser payload (per-frame joint
|
| 420 |
+
affines + root ground-track), tagged with the rig so the viewer loads the
|
| 421 |
+
correct skeleton/model."""
|
| 422 |
+
r = RIGS[rig_key]
|
| 423 |
+
out = _decode_motion(rig_key, motion_tensor)
|
| 424 |
+
posed = out["posed_joints"] # [T, J, 3] global joint positions
|
| 425 |
+
grm = out["global_rot_mats"] # [T, J, 3, 3] global rotations
|
| 426 |
+
if rig_key == "robot":
|
| 427 |
+
affines = _joint_transforms_robot(grm, posed)
|
| 428 |
+
else:
|
| 429 |
+
affines = _joint_affines_human(grm, posed)
|
| 430 |
+
return {
|
| 431 |
+
"rig": rig_key,
|
| 432 |
+
"fps": r["fps"],
|
| 433 |
+
"num_frames": int(posed.shape[0]),
|
| 434 |
+
"num_joints": int(posed.shape[1]),
|
| 435 |
+
"affines": affines, # drives the browser rig
|
| 436 |
+
"root": np.round(posed[:, r["root_idx"], :].astype(np.float32), 4).tolist(),
|
| 437 |
+
"prompt": prompt,
|
| 438 |
+
"seed": seed,
|
| 439 |
+
}
|
| 440 |
+
|
| 441 |
+
|
| 442 |
+
def _core_generate(rig, prompt, duration, diffusion_steps, cfg_weight, seed, randomize_seed, init_np):
|
| 443 |
+
rig_key = _normalize_rig(rig)
|
| 444 |
+
r = RIGS[rig_key]
|
| 445 |
+
prompt = (prompt or "").strip()
|
| 446 |
+
if not prompt:
|
| 447 |
+
raise gr.Error("Please enter a text prompt describing the motion.")
|
| 448 |
+
if randomize_seed:
|
| 449 |
+
seed = random.randint(0, MAX_SEED)
|
| 450 |
+
seed = int(seed)
|
| 451 |
+
seed_everything(seed)
|
| 452 |
+
steps = max(1, min(int(diffusion_steps), r["num_base_steps"]))
|
| 453 |
+
num_new = max(r["patch"], int(round(float(duration) * r["fps"])))
|
| 454 |
+
init_tensor = None if init_np is None else torch.from_numpy(init_np).to("cuda")
|
| 455 |
+
|
| 456 |
+
t0 = time.perf_counter()
|
| 457 |
+
with torch.no_grad():
|
| 458 |
+
full = _generate_sequence(rig_key, prompt, num_new, steps, cfg_weight, init_tensor)
|
| 459 |
+
payload = _pack_payload(rig_key, full, prompt, seed)
|
| 460 |
+
tag = "continue" if init_np is not None else "generate"
|
| 461 |
+
print(f"[{tag}:{rig_key}] '{prompt[:50]}' +{num_new}f -> {full.shape[1]}f total "
|
| 462 |
+
f"steps={steps} seed={seed} {time.perf_counter() - t0:.1f}s", flush=True)
|
| 463 |
+
return json.dumps(payload), seed, full.detach().cpu().numpy()
|
| 464 |
+
|
| 465 |
+
|
| 466 |
+
@spaces.GPU
|
| 467 |
+
def ui_generate(prompt, rig=DEFAULT_RIG, duration=5.0, diffusion_steps=NUM_BASE_STEPS,
|
| 468 |
+
cfg_weight=2.0, seed=0, randomize_seed=True):
|
| 469 |
+
"""Start a fresh clip (resets the running sequence)."""
|
| 470 |
+
return _core_generate(rig, prompt, duration, diffusion_steps, cfg_weight, seed, randomize_seed, None)
|
| 471 |
+
|
| 472 |
+
|
| 473 |
+
@spaces.GPU
|
| 474 |
+
def ui_continue(prompt, rig=DEFAULT_RIG, duration=5.0, diffusion_steps=NUM_BASE_STEPS,
|
| 475 |
+
cfg_weight=2.0, seed=0, randomize_seed=True, state=None):
|
| 476 |
+
"""Append a new action, continuing from the previous clip's final pose."""
|
| 477 |
+
return _core_generate(rig, prompt, duration, diffusion_steps, cfg_weight, seed, randomize_seed, state)
|
| 478 |
+
|
| 479 |
+
|
| 480 |
+
@spaces.GPU
|
| 481 |
+
def generate_motion(prompt: str, rig: str = DEFAULT_RIG, duration: float = 5.0,
|
| 482 |
+
diffusion_steps: int = NUM_BASE_STEPS, cfg_weight: float = 2.0,
|
| 483 |
+
seed: int = 0, randomize_seed: bool = True) -> tuple[str, int]:
|
| 484 |
+
"""Generate a 3D motion clip from a text prompt with ARDY.
|
| 485 |
+
|
| 486 |
+
Args:
|
| 487 |
+
prompt: Natural-language description of the motion (e.g. "a person walks in a circle").
|
| 488 |
+
rig: Which character to animate — "human" (27-joint skeleton) or "robot" (Unitree G1).
|
| 489 |
+
duration: Length of the generated motion in seconds.
|
| 490 |
+
diffusion_steps: Number of denoising steps (1..num_base_steps).
|
| 491 |
+
cfg_weight: Classifier-free-guidance weight for the text prompt.
|
| 492 |
+
seed: Random seed for reproducibility.
|
| 493 |
+
randomize_seed: If True, ignore `seed` and draw a fresh random one.
|
| 494 |
+
|
| 495 |
+
Returns:
|
| 496 |
+
A JSON string with the animated skeleton payload plus the seed used.
|
| 497 |
+
"""
|
| 498 |
+
payload_json, seed, _ = _core_generate(
|
| 499 |
+
rig, prompt, duration, diffusion_steps, cfg_weight, seed, randomize_seed, None
|
| 500 |
+
)
|
| 501 |
+
return payload_json, seed
|
| 502 |
+
|
| 503 |
+
|
| 504 |
+
@spaces.GPU
|
| 505 |
+
def generate_blender_motion(
|
| 506 |
+
prompt: str,
|
| 507 |
+
duration: float = 5.0,
|
| 508 |
+
diffusion_steps: int = NUM_BASE_STEPS,
|
| 509 |
+
cfg_weight: float = 2.0,
|
| 510 |
+
seed: int = 0,
|
| 511 |
+
randomize_seed: bool = True,
|
| 512 |
+
) -> tuple[str, str, str, int]:
|
| 513 |
+
"""Generate a human motion and return BVH, NPZ, and JSON metadata files."""
|
| 514 |
+
_, seed, motion_np = _core_generate(
|
| 515 |
+
"human",
|
| 516 |
+
prompt,
|
| 517 |
+
duration,
|
| 518 |
+
diffusion_steps,
|
| 519 |
+
cfg_weight,
|
| 520 |
+
seed,
|
| 521 |
+
randomize_seed,
|
| 522 |
+
None,
|
| 523 |
+
)
|
| 524 |
+
motion_tensor = torch.from_numpy(motion_np).to("cuda")
|
| 525 |
+
out = _decode_motion("human", motion_tensor)
|
| 526 |
+
r = RIGS["human"]
|
| 527 |
+
skeleton = r["skeleton"]
|
| 528 |
+
joint_names = list(skeleton.bone_order_names)
|
| 529 |
+
parents = list(r["parents"])
|
| 530 |
+
rest_positions = to_numpy(skeleton.neutral_joints)
|
| 531 |
+
|
| 532 |
+
output_dir = Path(tempfile.mkdtemp(prefix="ardy_blender_"))
|
| 533 |
+
npz_path = output_dir / "ardy_motion.npz"
|
| 534 |
+
bvh_path = output_dir / "ardy_motion.bvh"
|
| 535 |
+
metadata_path = output_dir / "metadata.json"
|
| 536 |
+
|
| 537 |
+
np.savez_compressed(
|
| 538 |
+
npz_path,
|
| 539 |
+
**out,
|
| 540 |
+
fps=np.asarray(r["fps"], dtype=np.float32),
|
| 541 |
+
text=np.asarray(prompt),
|
| 542 |
+
seed=np.asarray(seed, dtype=np.int64),
|
| 543 |
+
rig=np.asarray("human"),
|
| 544 |
+
joint_names=np.asarray(joint_names),
|
| 545 |
+
parents=np.asarray(parents, dtype=np.int32),
|
| 546 |
+
rest_positions=np.asarray(rest_positions, dtype=np.float32),
|
| 547 |
+
)
|
| 548 |
+
write_bvh(
|
| 549 |
+
bvh_path,
|
| 550 |
+
joint_names=joint_names,
|
| 551 |
+
parents=parents,
|
| 552 |
+
rest_positions=rest_positions,
|
| 553 |
+
local_rot_mats=out["local_rot_mats"],
|
| 554 |
+
root_positions=out["root_positions"],
|
| 555 |
+
fps=r["fps"],
|
| 556 |
+
)
|
| 557 |
+
metadata_path.write_text(
|
| 558 |
+
json.dumps(
|
| 559 |
+
{
|
| 560 |
+
"model": "nvidia/ARDY-Core-RP-20FPS-Horizon40",
|
| 561 |
+
"rig": "human",
|
| 562 |
+
"prompt": prompt,
|
| 563 |
+
"seed": seed,
|
| 564 |
+
"fps": r["fps"],
|
| 565 |
+
"frames": int(out["posed_joints"].shape[0]),
|
| 566 |
+
"joints": joint_names,
|
| 567 |
+
"parents": parents,
|
| 568 |
+
"coordinate_system": "Y-up, metric",
|
| 569 |
+
"post_processing": False,
|
| 570 |
+
},
|
| 571 |
+
ensure_ascii=False,
|
| 572 |
+
indent=2,
|
| 573 |
+
),
|
| 574 |
+
encoding="utf-8",
|
| 575 |
+
)
|
| 576 |
+
return str(bvh_path), str(npz_path), str(metadata_path), seed
|
| 577 |
+
|
| 578 |
+
|
| 579 |
+
# -----------------------------------------------------------------------------
|
| 580 |
+
# Front-end: a self-contained Three.js playground, delivered as a Gradio-native
|
| 581 |
+
# custom HTML component (Gradio 6 `gr.HTML` templates + js_on_load).
|
| 582 |
+
#
|
| 583 |
+
# The motion JSON is carried as the component's own `value` prop. `js_on_load`
|
| 584 |
+
# imports three.js, builds the scene once, wires the controls, then registers a
|
| 585 |
+
# `watch('value', ...)` callback that Gradio fires whenever the component is
|
| 586 |
+
# updated as the output of a Python event (Generate button / Examples).
|
| 587 |
+
#
|
| 588 |
+
# Each payload is tagged with its `rig`. The viewer ships the static data for
|
| 589 |
+
# BOTH rigs (human skin blob + G1 rigid-mesh blob) and switches at load time:
|
| 590 |
+
# - "human": one skinned body mesh (linear-blend skinning in the browser).
|
| 591 |
+
# - "robot": rigid per-joint G1 STL meshes posed by the joint transforms.
|
| 592 |
+
# -----------------------------------------------------------------------------
|
| 593 |
+
|
| 594 |
+
PLAYER_TEMPLATE = """
|
| 595 |
+
<div class="ardy-playground">
|
| 596 |
+
<div class="ardy-canvas-wrap">
|
| 597 |
+
<div class="ardy-hint">Generate a motion to load it into the playground.</div>
|
| 598 |
+
</div>
|
| 599 |
+
<div class="ardy-controls">
|
| 600 |
+
<button class="ardy-play ardy-btn" type="button">▶ Play</button>
|
| 601 |
+
<input class="ardy-scrub" type="range" min="0" max="0" value="0" step="1" />
|
| 602 |
+
<span class="ardy-frame">0 / 0</span>
|
| 603 |
+
<label class="ardy-lbl">Speed
|
| 604 |
+
<select class="ardy-speed">
|
| 605 |
+
<option value="0.25">0.25×</option>
|
| 606 |
+
<option value="0.5">0.5×</option>
|
| 607 |
+
<option value="1" selected>1×</option>
|
| 608 |
+
<option value="2">2×</option>
|
| 609 |
+
</select>
|
| 610 |
+
</label>
|
| 611 |
+
<label class="ardy-lbl"><input type="checkbox" class="ardy-loop" checked/> Loop</label>
|
| 612 |
+
<label class="ardy-lbl"><input type="checkbox" class="ardy-trail"/> Root trail</label>
|
| 613 |
+
</div>
|
| 614 |
+
<div class="ardy-caption"></div>
|
| 615 |
+
</div>
|
| 616 |
+
"""
|
| 617 |
+
|
| 618 |
+
# css_template rules are auto-scoped to this component by Gradio.
|
| 619 |
+
PLAYER_CSS_TEMPLATE = """
|
| 620 |
+
.ardy-playground { width: 100%; }
|
| 621 |
+
.ardy-canvas-wrap {
|
| 622 |
+
position: relative; width: 100%; height: 480px;
|
| 623 |
+
border-radius: 12px; overflow: hidden;
|
| 624 |
+
background: #ffffff;
|
| 625 |
+
border: 1px solid #e5e7eb;
|
| 626 |
+
}
|
| 627 |
+
.ardy-canvas-wrap canvas { display:block; width:100% !important; height:100% !important; }
|
| 628 |
+
.ardy-hint {
|
| 629 |
+
position:absolute; top:50%; left:50%; transform:translate(-50%,-50%);
|
| 630 |
+
color:#98a2b3; font-size:14px; text-align:center; pointer-events:none;
|
| 631 |
+
}
|
| 632 |
+
.ardy-controls {
|
| 633 |
+
display:flex; align-items:center; gap:12px; flex-wrap:wrap;
|
| 634 |
+
margin-top:10px; padding:8px 4px;
|
| 635 |
+
}
|
| 636 |
+
.ardy-controls .ardy-btn {
|
| 637 |
+
background:#76B900; color:#fff;
|
| 638 |
+
border:none; border-radius:8px; padding:6px 16px; cursor:pointer; font-weight:600;
|
| 639 |
+
}
|
| 640 |
+
.ardy-scrub { flex:1; min-width:160px; accent-color:#76B900; }
|
| 641 |
+
.ardy-frame { font-variant-numeric: tabular-nums; color: var(--body-text-color); min-width:70px; }
|
| 642 |
+
.ardy-lbl { font-size:13px; color: var(--body-text-color); display:flex; align-items:center; gap:4px; }
|
| 643 |
+
.ardy-caption { margin-top:6px; font-size:13px; color:#667085; }
|
| 644 |
+
"""
|
| 645 |
+
|
| 646 |
+
APP_CSS = """
|
| 647 |
+
#col-container { max-width: 1200px; margin: 0 auto; }
|
| 648 |
+
.dark .gradio-container { color: var(--body-text-color); }
|
| 649 |
+
"""
|
| 650 |
+
|
| 651 |
+
# Runs once, when the component first renders. `element`, `props`, and `watch`
|
| 652 |
+
# are injected by Gradio. We import three.js, decode the static skin data (human
|
| 653 |
+
# rig) and the static rigid-mesh data (robot rig), build the scene, and subscribe
|
| 654 |
+
# to value changes with `watch('value', ...)`. Each generated payload carries the
|
| 655 |
+
# per-frame joint affine matrices plus a `rig` tag; the viewer renders whichever
|
| 656 |
+
# rig the payload requests, swapping the on-screen mesh as needed.
|
| 657 |
+
PLAYER_JS_ON_LOAD = r"""
|
| 658 |
+
const root = element;
|
| 659 |
+
const q = (sel) => root.querySelector(sel);
|
| 660 |
+
|
| 661 |
+
const state = {
|
| 662 |
+
ready:false,
|
| 663 |
+
THREE:null, OrbitControls:null,
|
| 664 |
+
renderer:null, scene:null, camera:null, controls:null,
|
| 665 |
+
trailLine:null,
|
| 666 |
+
data:null, verts:null, frame:0, playing:false, lastT:0,
|
| 667 |
+
speed:1, loop:true, trail:false, pending:null,
|
| 668 |
+
rig:null, // which rig mesh is currently mounted in the scene
|
| 669 |
+
// human rig
|
| 670 |
+
skin:null, humanMesh:null, humanGeom:null,
|
| 671 |
+
// robot rig
|
| 672 |
+
robotMesh:null, robotGeom:null,
|
| 673 |
+
robotBaseVerts:null, robotFaces:null, robotVtot:0, robotParts:null,
|
| 674 |
+
};
|
| 675 |
+
|
| 676 |
+
// --- binary helpers ---------------------------------------------------------
|
| 677 |
+
function b64ToBytes(b64){
|
| 678 |
+
const bin = atob(b64); const out = new Uint8Array(bin.length);
|
| 679 |
+
for(let i=0;i<bin.length;i++) out[i]=bin.charCodeAt(i);
|
| 680 |
+
return out;
|
| 681 |
+
}
|
| 682 |
+
async function gunzip(bytes){
|
| 683 |
+
const ds = new DecompressionStream("gzip");
|
| 684 |
+
const buf = await new Response(new Blob([bytes]).stream().pipeThrough(ds)).arrayBuffer();
|
| 685 |
+
return buf;
|
| 686 |
+
}
|
| 687 |
+
|
| 688 |
+
// Decode the one-time static human skin blob into typed arrays.
|
| 689 |
+
async function decodeSkin(){
|
| 690 |
+
const buf = await gunzip(b64ToBytes(ARDY_SKIN_B64));
|
| 691 |
+
const V = ARDY_SKIN_META.V, F = ARDY_SKIN_META.F, W = ARDY_SKIN_META.W;
|
| 692 |
+
let o = 0;
|
| 693 |
+
const bindV = new Float32Array(buf.slice(o, o+V*3*4)); o += V*3*4;
|
| 694 |
+
const faces = new Uint32Array(buf.slice(o, o+F*3*4)); o += F*3*4;
|
| 695 |
+
const idx = new Uint8Array(buf.slice(o, o+V*W)); o += V*W;
|
| 696 |
+
const wt = new Float32Array(buf.slice(o, o+V*W*4)); o += V*W*4;
|
| 697 |
+
return {V, F, W, bindV, faces, idx, wt};
|
| 698 |
+
}
|
| 699 |
+
|
| 700 |
+
// Decode the one-time static robot rig blob: pre-transformed mesh vertices
|
| 701 |
+
// (joint-local frame) + global-indexed faces + per-mesh part table.
|
| 702 |
+
async function decodeRig(){
|
| 703 |
+
const buf = await gunzip(b64ToBytes(G1_MESH_B64));
|
| 704 |
+
const V = G1_MESH_META.Vtot, F = G1_MESH_META.Ftot;
|
| 705 |
+
let o = 0;
|
| 706 |
+
const baseVerts = new Float32Array(buf.slice(o, o+V*3*4)); o += V*3*4;
|
| 707 |
+
const faces = new Uint32Array(buf.slice(o, o+F*3*4)); o += F*3*4;
|
| 708 |
+
return {V, F, baseVerts, faces, parts: G1_MESH_META.parts};
|
| 709 |
+
}
|
| 710 |
+
|
| 711 |
+
// HUMAN: per-vertex linear-blend skinning for every frame (once per clip).
|
| 712 |
+
// A[f,j] is a 3x4 affine (row-major, 12 floats); posed vertex =
|
| 713 |
+
// sum_k w_k * A[idx_k] @ [bind_x, bind_y, bind_z, 1].
|
| 714 |
+
function skinAllFrames(A, T, J){
|
| 715 |
+
const s = state.skin, V = s.V, W = s.W, bindV = s.bindV, idx = s.idx, wt = s.wt;
|
| 716 |
+
const frames = new Array(T);
|
| 717 |
+
for(let f=0; f<T; f++){
|
| 718 |
+
const out = new Float32Array(V*3);
|
| 719 |
+
const Ab = f*J*12;
|
| 720 |
+
for(let v=0; v<V; v++){
|
| 721 |
+
const bx = bindV[v*3], by = bindV[v*3+1], bz = bindV[v*3+2];
|
| 722 |
+
let x=0, y=0, z=0;
|
| 723 |
+
for(let k=0; k<W; k++){
|
| 724 |
+
const w = wt[v*W+k]; if(w===0) continue;
|
| 725 |
+
const a = Ab + idx[v*W+k]*12;
|
| 726 |
+
x += w*(A[a]*bx + A[a+1]*by + A[a+2]*bz + A[a+3]);
|
| 727 |
+
y += w*(A[a+4]*bx + A[a+5]*by + A[a+6]*bz + A[a+7]);
|
| 728 |
+
z += w*(A[a+8]*bx + A[a+9]*by + A[a+10]*bz + A[a+11]);
|
| 729 |
+
}
|
| 730 |
+
out[v*3]=x; out[v*3+1]=y; out[v*3+2]=z;
|
| 731 |
+
}
|
| 732 |
+
frames[f] = out;
|
| 733 |
+
}
|
| 734 |
+
return frames;
|
| 735 |
+
}
|
| 736 |
+
|
| 737 |
+
// ROBOT: per-frame rigid transform: for every mesh part, world_v = pos + R @ v'
|
| 738 |
+
// where (R,pos) is the driving joint's global transform this frame and v' is
|
| 739 |
+
// the vertex already baked into that joint's local frame.
|
| 740 |
+
function poseAllFrames(A, T, J){
|
| 741 |
+
const V = state.robotVtot, base = state.robotBaseVerts, parts = state.robotParts;
|
| 742 |
+
const frames = new Array(T);
|
| 743 |
+
for(let f=0; f<T; f++){
|
| 744 |
+
const out = new Float32Array(V*3);
|
| 745 |
+
const Ab = f*J*12;
|
| 746 |
+
for(let p=0; p<parts.length; p++){
|
| 747 |
+
const jp = parts[p];
|
| 748 |
+
const a = Ab + jp.joint*12;
|
| 749 |
+
const r0=A[a], r1=A[a+1], r2=A[a+2], px=A[a+3];
|
| 750 |
+
const r3=A[a+4], r4=A[a+5], r5=A[a+6], py=A[a+7];
|
| 751 |
+
const r6=A[a+8], r7=A[a+9], r8=A[a+10], pz=A[a+11];
|
| 752 |
+
const vs = jp.v_off, ve = jp.v_off + jp.v_cnt;
|
| 753 |
+
for(let v=vs; v<ve; v++){
|
| 754 |
+
const bx=base[v*3], by=base[v*3+1], bz=base[v*3+2];
|
| 755 |
+
out[v*3] = px + r0*bx + r1*by + r2*bz;
|
| 756 |
+
out[v*3+1] = py + r3*bx + r4*by + r5*bz;
|
| 757 |
+
out[v*3+2] = pz + r6*bx + r7*by + r8*bz;
|
| 758 |
+
}
|
| 759 |
+
}
|
| 760 |
+
frames[f] = out;
|
| 761 |
+
}
|
| 762 |
+
return frames;
|
| 763 |
+
}
|
| 764 |
+
|
| 765 |
+
// --- three.js scene ---------------------------------------------------------
|
| 766 |
+
function initScene(){
|
| 767 |
+
const THREE = state.THREE, OrbitControls = state.OrbitControls;
|
| 768 |
+
const wrap = q(".ardy-canvas-wrap");
|
| 769 |
+
if(!wrap || state.renderer) return;
|
| 770 |
+
const w = wrap.clientWidth || 800, h = wrap.clientHeight || 480;
|
| 771 |
+
const scene = new THREE.Scene(); scene.background = new THREE.Color(0xffffff);
|
| 772 |
+
const camera = new THREE.PerspectiveCamera(42, w/h, 0.05, 200);
|
| 773 |
+
camera.position.set(3.8, 2.2, 4.7);
|
| 774 |
+
const renderer = new THREE.WebGLRenderer({antialias:true});
|
| 775 |
+
renderer.setSize(w, h); renderer.setPixelRatio(Math.min(window.devicePixelRatio,2));
|
| 776 |
+
renderer.shadowMap.enabled = true; renderer.shadowMap.type = THREE.PCFSoftShadowMap;
|
| 777 |
+
wrap.appendChild(renderer.domElement);
|
| 778 |
+
const controls = new OrbitControls(camera, renderer.domElement);
|
| 779 |
+
controls.target.set(0, 0.9, 0); controls.enableDamping = true;
|
| 780 |
+
|
| 781 |
+
scene.add(new THREE.HemisphereLight(0xffffff, 0xdfe4ee, 1.4));
|
| 782 |
+
const key = new THREE.DirectionalLight(0xffffff, 1.5);
|
| 783 |
+
key.position.set(3, 6, 4); key.castShadow = true;
|
| 784 |
+
key.shadow.mapSize.set(2048, 2048);
|
| 785 |
+
const c = key.shadow.camera; c.near=0.5; c.far=25; c.left=-3; c.right=3; c.top=3; c.bottom=-3;
|
| 786 |
+
key.shadow.bias = -0.0004;
|
| 787 |
+
scene.add(key);
|
| 788 |
+
scene.add(new THREE.DirectionalLight(0xeef2ff, 0.35).translateX(-4).translateZ(-2));
|
| 789 |
+
|
| 790 |
+
const ground = new THREE.Mesh(
|
| 791 |
+
new THREE.PlaneGeometry(40, 40),
|
| 792 |
+
new THREE.ShadowMaterial({opacity:0.16})
|
| 793 |
+
);
|
| 794 |
+
ground.rotation.x = -Math.PI/2; ground.position.y = 0; ground.receiveShadow = true;
|
| 795 |
+
scene.add(ground);
|
| 796 |
+
|
| 797 |
+
const grid = new THREE.GridHelper(10, 20, 0xc4c9d4, 0xe4e7ee);
|
| 798 |
+
grid.position.y = 0.0015; scene.add(grid);
|
| 799 |
+
|
| 800 |
+
state.renderer=renderer; state.scene=scene; state.camera=camera; state.controls=controls;
|
| 801 |
+
|
| 802 |
+
new ResizeObserver(()=>{
|
| 803 |
+
const w2 = wrap.clientWidth, h2 = wrap.clientHeight;
|
| 804 |
+
if(w2>0 && h2>0){ camera.aspect=w2/h2; camera.updateProjectionMatrix(); renderer.setSize(w2,h2); }
|
| 805 |
+
}).observe(wrap);
|
| 806 |
+
|
| 807 |
+
animate();
|
| 808 |
+
}
|
| 809 |
+
|
| 810 |
+
// Mount the mesh for the requested rig (lazily built, then shown/hidden). Only
|
| 811 |
+
// one rig mesh is visible at a time; both share the scene once created.
|
| 812 |
+
function mountRig(rig){
|
| 813 |
+
const THREE = state.THREE;
|
| 814 |
+
if(rig === "robot"){
|
| 815 |
+
if(!state.robotMesh){
|
| 816 |
+
const geom = new THREE.BufferGeometry();
|
| 817 |
+
geom.setIndex(new THREE.BufferAttribute(state.robotFaces, 1));
|
| 818 |
+
geom.setAttribute("position", new THREE.BufferAttribute(new Float32Array(state.robotVtot*3), 3));
|
| 819 |
+
const mat = new THREE.MeshStandardMaterial({color:0xd7dde6, roughness:0.5, metalness:0.55});
|
| 820 |
+
const mesh = new THREE.Mesh(geom, mat);
|
| 821 |
+
mesh.castShadow = true; mesh.frustumCulled = false;
|
| 822 |
+
state.scene.add(mesh); state.robotMesh = mesh; state.robotGeom = geom;
|
| 823 |
+
}
|
| 824 |
+
if(state.humanMesh) state.humanMesh.visible = false;
|
| 825 |
+
state.robotMesh.visible = true;
|
| 826 |
+
return state.robotGeom;
|
| 827 |
+
} else {
|
| 828 |
+
if(!state.humanMesh){
|
| 829 |
+
const s = state.skin;
|
| 830 |
+
const geom = new THREE.BufferGeometry();
|
| 831 |
+
geom.setIndex(new THREE.BufferAttribute(s.faces, 1));
|
| 832 |
+
geom.setAttribute("position", new THREE.BufferAttribute(new Float32Array(s.V*3), 3));
|
| 833 |
+
const mat = new THREE.MeshStandardMaterial({color:0x98bdff, roughness:0.85, metalness:0.0});
|
| 834 |
+
const mesh = new THREE.Mesh(geom, mat);
|
| 835 |
+
mesh.castShadow = true; mesh.frustumCulled = false;
|
| 836 |
+
state.scene.add(mesh); state.humanMesh = mesh; state.humanGeom = geom;
|
| 837 |
+
}
|
| 838 |
+
if(state.robotMesh) state.robotMesh.visible = false;
|
| 839 |
+
state.humanMesh.visible = true;
|
| 840 |
+
return state.humanGeom;
|
| 841 |
+
}
|
| 842 |
+
}
|
| 843 |
+
|
| 844 |
+
function activeGeom(){
|
| 845 |
+
return (state.rig === "robot") ? state.robotGeom : state.humanGeom;
|
| 846 |
+
}
|
| 847 |
+
|
| 848 |
+
function setFrame(f){
|
| 849 |
+
if(!state.verts) return;
|
| 850 |
+
const T = state.data.num_frames;
|
| 851 |
+
f = Math.max(0, Math.min(T-1, f|0));
|
| 852 |
+
state.frame = f;
|
| 853 |
+
const geom = activeGeom();
|
| 854 |
+
if(!geom) return;
|
| 855 |
+
const pos = geom.getAttribute("position");
|
| 856 |
+
pos.array.set(state.verts[f]);
|
| 857 |
+
pos.needsUpdate = true;
|
| 858 |
+
geom.computeVertexNormals();
|
| 859 |
+
geom.computeBoundingSphere();
|
| 860 |
+
updateTrail();
|
| 861 |
+
const scrub = q(".ardy-scrub"); if(scrub) scrub.value = f;
|
| 862 |
+
const lbl = q(".ardy-frame"); if(lbl) lbl.textContent = (f+1)+" / "+T;
|
| 863 |
+
}
|
| 864 |
+
|
| 865 |
+
function updateTrail(){
|
| 866 |
+
const THREE = state.THREE;
|
| 867 |
+
if(!state.data || !state.data.root){ if(state.trailLine) state.trailLine.visible=false; return; }
|
| 868 |
+
if(!state.trail){ if(state.trailLine) state.trailLine.visible=false; return; }
|
| 869 |
+
const T = state.data.num_frames, root = state.data.root;
|
| 870 |
+
if(!state.trailLine){
|
| 871 |
+
const g = new THREE.BufferGeometry();
|
| 872 |
+
g.setAttribute("position", new THREE.BufferAttribute(new Float32Array(T*3),3));
|
| 873 |
+
state.trailLine = new THREE.Line(g, new THREE.LineBasicMaterial({color:0xf59e0b}));
|
| 874 |
+
state.scene.add(state.trailLine);
|
| 875 |
+
}
|
| 876 |
+
state.trailLine.visible = true;
|
| 877 |
+
const attr = state.trailLine.geometry.getAttribute("position");
|
| 878 |
+
for(let t=0;t<T;t++){ attr.setXYZ(t, root[t][0], 0.006, root[t][2]); }
|
| 879 |
+
attr.needsUpdate = true;
|
| 880 |
+
state.trailLine.geometry.setDrawRange(0, Math.max(1, state.frame+1));
|
| 881 |
+
}
|
| 882 |
+
|
| 883 |
+
function animate(){
|
| 884 |
+
requestAnimationFrame(animate);
|
| 885 |
+
if(!state.renderer) return;
|
| 886 |
+
const now = performance.now();
|
| 887 |
+
if(state.playing && state.verts){
|
| 888 |
+
const dt = (now - state.lastT)/1000;
|
| 889 |
+
const fps = state.data.fps * state.speed;
|
| 890 |
+
if(dt >= 1/Math.max(1e-3,fps)){
|
| 891 |
+
state.lastT = now;
|
| 892 |
+
let nf = state.frame + 1;
|
| 893 |
+
if(nf >= state.data.num_frames){
|
| 894 |
+
if(state.loop){ nf = 0; } else { nf = state.data.num_frames-1; setPlaying(false); }
|
| 895 |
+
}
|
| 896 |
+
setFrame(nf);
|
| 897 |
+
}
|
| 898 |
+
}
|
| 899 |
+
state.controls.update();
|
| 900 |
+
state.renderer.render(state.scene, state.camera);
|
| 901 |
+
}
|
| 902 |
+
|
| 903 |
+
function setPlaying(p){
|
| 904 |
+
state.playing = p;
|
| 905 |
+
const b = q(".ardy-play");
|
| 906 |
+
if(b) b.textContent = p ? "⏸ Pause" : "▶ Play";
|
| 907 |
+
state.lastT = performance.now();
|
| 908 |
+
}
|
| 909 |
+
|
| 910 |
+
function loadData(data){
|
| 911 |
+
initScene();
|
| 912 |
+
const rig = (data.rig === "robot") ? "robot" : "human";
|
| 913 |
+
state.rig = rig;
|
| 914 |
+
mountRig(rig);
|
| 915 |
+
state.data = data;
|
| 916 |
+
// Decode per-frame affines and pre-pose every frame (one-time cost per clip).
|
| 917 |
+
const T = data.num_frames, J = data.num_joints;
|
| 918 |
+
const Abytes = b64ToBytes(data.affines);
|
| 919 |
+
const A = new Float32Array(Abytes.buffer, Abytes.byteOffset, Abytes.byteLength/4);
|
| 920 |
+
state.verts = (rig === "robot") ? poseAllFrames(A, T, J) : skinAllFrames(A, T, J);
|
| 921 |
+
if(state.trailLine){ state.scene.remove(state.trailLine); state.trailLine.geometry.dispose(); state.trailLine=null; }
|
| 922 |
+
const scrub = q(".ardy-scrub"); if(scrub){ scrub.max = T-1; scrub.value = 0; }
|
| 923 |
+
const hint = q(".ardy-hint"); if(hint) hint.style.display = "none";
|
| 924 |
+
const cap = q(".ardy-caption");
|
| 925 |
+
if(cap) cap.textContent = '"' + data.prompt + '" · ' + (rig==="robot"?"robot":"human") +
|
| 926 |
+
' · ' + T + ' frames @ ' + data.fps + ' fps · seed ' + data.seed;
|
| 927 |
+
root.dataset.ardyLoaded = "1";
|
| 928 |
+
root.dataset.ardyRig = rig;
|
| 929 |
+
root.dataset.ardyFrames = String(T);
|
| 930 |
+
setFrame(0);
|
| 931 |
+
setPlaying(true);
|
| 932 |
+
}
|
| 933 |
+
|
| 934 |
+
function applyPayload(payload){
|
| 935 |
+
if(!payload) return;
|
| 936 |
+
if(!state.ready){ state.pending = payload; return; } // three.js / rigs still loading
|
| 937 |
+
try { loadData(JSON.parse(payload)); }
|
| 938 |
+
catch(e){ console.error("ARDY playground load error", e); }
|
| 939 |
+
}
|
| 940 |
+
|
| 941 |
+
function wireControls(){
|
| 942 |
+
const bind = (sel, ev, fn) => {
|
| 943 |
+
const el = q(sel); if(el && !el.dataset.wired){ el.dataset.wired="1"; el.addEventListener(ev, fn); }
|
| 944 |
+
};
|
| 945 |
+
bind(".ardy-play", "click", ()=> setPlaying(!state.playing));
|
| 946 |
+
bind(".ardy-scrub", "input", (e)=>{ setPlaying(false); setFrame(parseInt(e.target.value)); });
|
| 947 |
+
bind(".ardy-speed", "change", (e)=>{ state.speed = parseFloat(e.target.value); });
|
| 948 |
+
bind(".ardy-loop", "change", (e)=>{ state.loop = e.target.checked; });
|
| 949 |
+
bind(".ardy-trail", "change", (e)=>{ state.trail = e.target.checked; updateTrail(); });
|
| 950 |
+
}
|
| 951 |
+
|
| 952 |
+
// Gradio-native hand-off: render whenever the component's value prop updates as
|
| 953 |
+
// the output of a Python event (Generate / Continue / Examples).
|
| 954 |
+
if (typeof watch === "function") {
|
| 955 |
+
watch("value", () => applyPayload(props.value));
|
| 956 |
+
}
|
| 957 |
+
|
| 958 |
+
// Import three.js (esm.sh, not jsDelivr: the OrbitControls addon has an internal
|
| 959 |
+
// bare `import ... from "three"` that a browser dynamic import() can't resolve
|
| 960 |
+
// without an import map; esm.sh rewrites it and dedupes three) and decode both
|
| 961 |
+
// rigs, then flush any value that already arrived.
|
| 962 |
+
Promise.all([
|
| 963 |
+
import("https://esm.sh/three@0.160.0"),
|
| 964 |
+
import("https://esm.sh/three@0.160.0/examples/jsm/controls/OrbitControls.js"),
|
| 965 |
+
decodeSkin(),
|
| 966 |
+
decodeRig(),
|
| 967 |
+
]).then(([THREE, oc, skin, rig]) => {
|
| 968 |
+
state.THREE = THREE;
|
| 969 |
+
state.OrbitControls = oc.OrbitControls;
|
| 970 |
+
state.skin = skin;
|
| 971 |
+
state.robotBaseVerts = rig.baseVerts;
|
| 972 |
+
state.robotFaces = rig.faces;
|
| 973 |
+
state.robotVtot = rig.V;
|
| 974 |
+
state.robotParts = rig.parts;
|
| 975 |
+
state.ready = true;
|
| 976 |
+
wireControls();
|
| 977 |
+
initScene();
|
| 978 |
+
const start = state.pending || props.value;
|
| 979 |
+
if (start) applyPayload(start);
|
| 980 |
+
}).catch((e)=> console.error("ARDY viewer init failed", e));
|
| 981 |
+
"""
|
| 982 |
+
|
| 983 |
+
EXAMPLES = [
|
| 984 |
+
["A person walks forward confidently.", "human", 5.0],
|
| 985 |
+
["A person walks in a circle.", "human", 6.0],
|
| 986 |
+
["A person jumps up and down.", "human", 4.0],
|
| 987 |
+
["The robot walks forward confidently.", "robot", 5.0],
|
| 988 |
+
["The robot waves with the right hand.", "robot", 4.0],
|
| 989 |
+
["The robot crouches down and then stands back up.", "robot", 5.0],
|
| 990 |
+
]
|
| 991 |
+
|
| 992 |
+
with gr.Blocks() as demo:
|
| 993 |
+
with gr.Column(elem_id="col-container"):
|
| 994 |
+
gr.Markdown(
|
| 995 |
+
"""
|
| 996 |
+
# 🕺 ARDY Motion Playground
|
| 997 |
+
Interactive text-to-motion with **[ARDY](https://research.nvidia.com/labs/sil/projects/ardy/)**
|
| 998 |
+
(Autoregressive Diffusion with Hybrid Representation) by NVIDIA.
|
| 999 |
+
Pick a **rig** (human or robot), type a prompt and **Generate** a 3D motion clip,
|
| 1000 |
+
then **orbit, scrub, and play** it below.
|
| 1001 |
+
Chain actions with **Continue ▸** — the same character keeps going from where it stopped.
|
| 1002 |
+
"""
|
| 1003 |
+
)
|
| 1004 |
+
|
| 1005 |
+
# Running latent state (normalized feature tensor, CPU) — lets "Continue"
|
| 1006 |
+
# resume the same character from the end of the previous clip.
|
| 1007 |
+
seq_state = gr.State(None)
|
| 1008 |
+
|
| 1009 |
+
prompt = gr.Textbox(
|
| 1010 |
+
label="Motion prompt",
|
| 1011 |
+
placeholder="e.g. a person walks in a circle then waves",
|
| 1012 |
+
lines=2,
|
| 1013 |
+
)
|
| 1014 |
+
rig = gr.Radio(
|
| 1015 |
+
choices=[("🕺 Human", "human"), ("🤖 Robot (Unitree G1)", "robot")],
|
| 1016 |
+
value=DEFAULT_RIG,
|
| 1017 |
+
label="Rig",
|
| 1018 |
+
)
|
| 1019 |
+
with gr.Row():
|
| 1020 |
+
run = gr.Button("Generate", variant="primary", scale=2)
|
| 1021 |
+
cont = gr.Button("Continue ▸", variant="secondary", scale=1)
|
| 1022 |
+
|
| 1023 |
+
# The interactive 3D playground — a Gradio-native custom HTML component.
|
| 1024 |
+
# Its `value` (the motion JSON) is set directly by the Generate handler;
|
| 1025 |
+
# `watch('value', ...)` in js_on_load renders it. Ship the static data for
|
| 1026 |
+
# both rigs (human skin blob + G1 rigid-mesh blob) once, as a header
|
| 1027 |
+
# prepended to js_on_load; the per-frame affines ride in each payload.
|
| 1028 |
+
_rig_header = (
|
| 1029 |
+
f'const ARDY_SKIN_B64="{SKIN_B64}";\n'
|
| 1030 |
+
f"const ARDY_SKIN_META={json.dumps(SKIN_META)};\n"
|
| 1031 |
+
f'const G1_MESH_B64="{G1_MESH_B64}";\n'
|
| 1032 |
+
f"const G1_MESH_META={json.dumps(G1_MESH_META)};\n"
|
| 1033 |
+
)
|
| 1034 |
+
player = gr.HTML(
|
| 1035 |
+
value="",
|
| 1036 |
+
html_template=PLAYER_TEMPLATE,
|
| 1037 |
+
css_template=PLAYER_CSS_TEMPLATE,
|
| 1038 |
+
js_on_load=_rig_header + PLAYER_JS_ON_LOAD,
|
| 1039 |
+
elem_id="ardy-player",
|
| 1040 |
+
)
|
| 1041 |
+
|
| 1042 |
+
with gr.Accordion("Advanced settings", open=False):
|
| 1043 |
+
duration = gr.Slider(1.0, 10.0, value=5.0, step=0.5, label="Duration (seconds)")
|
| 1044 |
+
diffusion_steps = gr.Slider(
|
| 1045 |
+
1, NUM_BASE_STEPS, value=NUM_BASE_STEPS, step=1, label="Diffusion steps"
|
| 1046 |
+
)
|
| 1047 |
+
cfg_weight = gr.Slider(1.0, 6.0, value=2.0, step=0.5, label="Text guidance (CFG)")
|
| 1048 |
+
with gr.Row():
|
| 1049 |
+
randomize_seed = gr.Checkbox(label="Randomize seed", value=True)
|
| 1050 |
+
seed = gr.Number(label="Seed", value=0, precision=0)
|
| 1051 |
+
|
| 1052 |
+
with gr.Accordion("Blender export", open=False):
|
| 1053 |
+
gr.Markdown(
|
| 1054 |
+
"Generate the human rig as **BVH**, full-fidelity **NPZ**, and "
|
| 1055 |
+
"JSON metadata. Import the BVH directly in Blender."
|
| 1056 |
+
)
|
| 1057 |
+
export_blender = gr.Button("Generate Blender files", variant="secondary")
|
| 1058 |
+
with gr.Row():
|
| 1059 |
+
bvh_file = gr.File(label="Blender BVH")
|
| 1060 |
+
npz_file = gr.File(label="ARDY NPZ")
|
| 1061 |
+
metadata_file = gr.File(label="Metadata")
|
| 1062 |
+
|
| 1063 |
+
gr.Examples(
|
| 1064 |
+
examples=EXAMPLES,
|
| 1065 |
+
inputs=[prompt, rig, duration],
|
| 1066 |
+
outputs=[player, seed, seq_state],
|
| 1067 |
+
fn=ui_generate,
|
| 1068 |
+
cache_examples=False,
|
| 1069 |
+
run_on_click=True,
|
| 1070 |
+
)
|
| 1071 |
+
|
| 1072 |
+
gr.Markdown(
|
| 1073 |
+
"""
|
| 1074 |
+
<small>Rigs: **ARDY-Core-RP-20FPS-Horizon40** (human, 27-joint skeleton @ 20 fps)
|
| 1075 |
+
and **ARDY-G1-RP-25FPS-Horizon52** (Unitree G1 robot, 34-joint skeleton @ 25 fps).
|
| 1076 |
+
Text encoder: LLM2Vec-Llama-3-8B. Post-processing (foot-skate cleanup) is disabled in this demo.
|
| 1077 |
+
Motion is generated autoregressively; longer clips take longer.
|
| 1078 |
+
**Generate** starts a new clip; **Continue ▸** keeps the same character going,
|
| 1079 |
+
transitioning it into the new prompt (like the reference demo's prompt timeline).</small>
|
| 1080 |
+
"""
|
| 1081 |
+
)
|
| 1082 |
+
|
| 1083 |
+
_gen_inputs = [prompt, rig, duration, diffusion_steps, cfg_weight, seed, randomize_seed]
|
| 1084 |
+
# Generate starts fresh; Continue resumes from the running latent state.
|
| 1085 |
+
# The payload is written straight into the player's `value`; its js_on_load
|
| 1086 |
+
# `watch('value', ...)` renders it (loading the correct rig from the payload).
|
| 1087 |
+
run.click(fn=ui_generate, inputs=_gen_inputs,
|
| 1088 |
+
outputs=[player, seed, seq_state], api_name=False)
|
| 1089 |
+
cont.click(fn=ui_continue, inputs=_gen_inputs + [seq_state],
|
| 1090 |
+
outputs=[player, seed, seq_state], api_name=False)
|
| 1091 |
+
export_blender.click(
|
| 1092 |
+
fn=generate_blender_motion,
|
| 1093 |
+
inputs=[prompt, duration, diffusion_steps, cfg_weight, seed, randomize_seed],
|
| 1094 |
+
outputs=[bvh_file, npz_file, metadata_file, seed],
|
| 1095 |
+
api_name=False,
|
| 1096 |
+
)
|
| 1097 |
+
|
| 1098 |
+
# Clean single-shot endpoint for the HTTP API / MCP tool (no session state).
|
| 1099 |
+
gr.api(generate_motion, api_name="generate")
|
| 1100 |
+
gr.api(generate_blender_motion, api_name="generate_blender")
|
| 1101 |
+
|
| 1102 |
+
demo.queue()
|
| 1103 |
+
|
| 1104 |
+
if __name__ == "__main__":
|
| 1105 |
+
# Gradio 6 moved theme/css from the Blocks constructor to launch(). The
|
| 1106 |
+
# player's JS/CSS now live on the gr.HTML component itself (js_on_load /
|
| 1107 |
+
# css_template), so no global `head=` script is needed.
|
| 1108 |
+
demo.launch(
|
| 1109 |
+
theme=gr.themes.Citrus(),
|
| 1110 |
+
css=APP_CSS,
|
| 1111 |
+
mcp_server=True,
|
| 1112 |
+
ssr_mode=False,
|
| 1113 |
+
)
|
ardy/__init__.py
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
| 2 |
+
# SPDX-License-Identifier: Apache-2.0
|
| 3 |
+
|
| 4 |
+
"""ARDY: text-to-motion generation."""
|
ardy/assets.py
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
| 2 |
+
# SPDX-License-Identifier: Apache-2.0
|
| 3 |
+
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
PACKAGE_ROOT = Path(__file__).resolve().parent
|
| 7 |
+
ASSETS_ROOT = PACKAGE_ROOT / "assets"
|
| 8 |
+
DEMO_ASSETS_ROOT = ASSETS_ROOT / "demo"
|
| 9 |
+
DEMO_EXAMPLES_ROOT = DEMO_ASSETS_ROOT / "examples"
|
| 10 |
+
SKELETONS_ROOT = ASSETS_ROOT / "skeletons"
|
| 11 |
+
SOMA_ASSETS_ROOT = ASSETS_ROOT / "SOMA"
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def skeleton_asset_path(*parts: str) -> Path:
|
| 15 |
+
return SKELETONS_ROOT.joinpath(*parts)
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def demo_asset_path(*parts: str) -> Path:
|
| 19 |
+
return DEMO_ASSETS_ROOT.joinpath(*parts)
|
ardy/assets/skeletons/cskel27/joints.p
ADDED
|
Binary file (1.82 kB). View file
|
|
|
ardy/assets/skeletons/cskel27/skin_standard.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4ed2b1ea33a997d7777a59d74ec2312b52af2276331048c954441f5f44f6e58b
|
| 3 |
+
size 237170
|
ardy/assets/skeletons/cskel29/joints.p
ADDED
|
Binary file (1.06 kB). View file
|
|
|
ardy/assets/skeletons/cskel29/standard_t_pose_cskel29_joint_quat.p
ADDED
|
Binary file (1.78 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/joints.p
ADDED
|
Binary file (1.94 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/head_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:005fb67fbd3eff94aa8bf4a6e83238174e9f91b6721f7111594322f223724411
|
| 3 |
+
size 932784
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_ankle_pitch_link.STL
ADDED
|
Binary file (71.2 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_ankle_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c4092af943141d4d9f74232f3cfa345afc6565f46a077793b8ae0e68b39dc33f
|
| 3 |
+
size 653384
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_elbow_link.STL
ADDED
|
Binary file (88.8 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_pitch_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4725168105ee768ee31638ef22b53f6be2d7641bfd7cfefe803488d884776fa4
|
| 3 |
+
size 181684
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:91f25922ee8a7c3152790051bebad17b4d9cd243569c38fe340285ff93a97acf
|
| 3 |
+
size 192184
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_hip_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a16d88aa6ddac8083aa7ad55ed317bea44b1fa003d314fba88965b7ed0f3b55b
|
| 3 |
+
size 296284
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_knee_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d92b9e3d3a636761150bb8025e32514c4602b91c7028d523ee42b3e632de477
|
| 3 |
+
size 854884
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_rubber_hand.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cff2221a690fa69303f61fce68f2d155c1517b52efb6ca9262dd56e0bc6e70fe
|
| 3 |
+
size 2287484
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_pitch_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f0d1cfd02fcf0d42f95e678eeca33da3afbcc366ffba5c052847773ec4f31d52
|
| 3 |
+
size 176784
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb9df21687773522598dc384f1a2945c7519f11cbc8bd372a49170316d6eee88
|
| 3 |
+
size 400284
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_shoulder_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1aa97e9748e924336567992181f78c7cd0652fd52a4afcca3df6b2ef6f9e712e
|
| 3 |
+
size 249184
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_pitch_link.STL
ADDED
|
Binary file (86 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:edc387c9a0ba8c2237e9b296d32531426fabeb6f53e58df45c76106bca74148c
|
| 3 |
+
size 356184
|
ardy/assets/skeletons/g1skel34/meshes/g1/left_wrist_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:83f8fb3a726bf9613d65dd14f0f447cb918c3c95b3938042a0c9c09749267d3b
|
| 3 |
+
size 318684
|
ardy/assets/skeletons/g1skel34/meshes/g1/logo_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8571a0a19bc4916fa55f91449f51d5fdefd751000054865a842449429d5f155b
|
| 3 |
+
size 243384
|
ardy/assets/skeletons/g1skel34/meshes/g1/pelvis.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5ba6bbc888e630550140d3c26763f10206da8c8bd30ed886b8ede41c61f57a31
|
| 3 |
+
size 1060884
|
ardy/assets/skeletons/g1skel34/meshes/g1/pelvis_contour_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5cc5c2c7a312329e3feeb2b03d3fc09fc29705bd01864f6767e51be959662420
|
| 3 |
+
size 1805184
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_ankle_pitch_link.STL
ADDED
|
Binary file (71.2 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_ankle_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4b66222ea56653e627711b56d0a8949b4920da5df091da0ceb343f54e884e3a5
|
| 3 |
+
size 653784
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_elbow_link.STL
ADDED
|
Binary file (88.8 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_pitch_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e4f3c99d7f4a7d34eadbef9461fc66e3486cb5442db1ec50c86317d459f1a9c6
|
| 3 |
+
size 181284
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c254ef66a356f492947f360dd931965477b631e3fcc841f91ccc46d945d54f6
|
| 3 |
+
size 192684
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_hip_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e479c2936ca2057e9eb2f7dff6c189b7419d7b8484dea0b298cbb36a2a6aa668
|
| 3 |
+
size 296284
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_knee_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:63c4008449c9bbe701a6e2b557b7a252e90cf3a5abcf54cee46862b9a69f8ec8
|
| 3 |
+
size 852284
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_rubber_hand.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:99533b778bca6246144fa511bb9d4e555e075c641f2a0251e04372869cd99d67
|
| 3 |
+
size 2192684
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_pitch_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:24cdb387e0128dfe602770a81c56cdce3a0181d34d039a11d1aaf8819b7b8c02
|
| 3 |
+
size 176784
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:962b97c48f9ce9e8399f45dd9522e0865d19aa9fd299406b2d475a8fc4a53e81
|
| 3 |
+
size 401884
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_shoulder_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a0b76489271da0c72461a344c9ffb0f0c6e64f019ea5014c1624886c442a2fe5
|
| 3 |
+
size 249984
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_pitch_link.STL
ADDED
|
Binary file (79.6 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_roll_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a7ee9212ff5b94d6cb7f52bb1bbf3f352194d5b598acff74f4c77d340c5b344f
|
| 3 |
+
size 356084
|
ardy/assets/skeletons/g1skel34/meshes/g1/right_wrist_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc9dece2d12509707e0057ba2e48df8f3d56db0c79410212963a25e8a50f61a6
|
| 3 |
+
size 341484
|
ardy/assets/skeletons/g1skel34/meshes/g1/torso_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e96d023f0368a4e3450b86ca5d4f10227d8141156a373e7da8cb3c93266523e0
|
| 3 |
+
size 2232984
|
ardy/assets/skeletons/g1skel34/meshes/g1/torso_link_rev_1_0.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:11ddb46f2098efbbd8816b1d65893632d8e78be936376c7cdcd6771899ccc723
|
| 3 |
+
size 2570584
|
ardy/assets/skeletons/g1skel34/meshes/g1/waist_roll_link.STL
ADDED
|
Binary file (24.2 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/waist_roll_link_rev_1_0.STL
ADDED
|
Binary file (85.9 kB). View file
|
|
|
ardy/assets/skeletons/g1skel34/meshes/g1/waist_yaw_link.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2883f20e03f09b669b5b4ce10677ee6b5191c0934b584d7cbaef2d0662856ffb
|
| 3 |
+
size 336284
|
ardy/assets/skeletons/g1skel34/meshes/g1/waist_yaw_link_rev_1_0.STL
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ec6db442b11f25eed898b5add07940c85d804f300de24dcbd264ccd8be7d554c
|
| 3 |
+
size 619984
|
ardy/assets/skeletons/g1skel34/xml/g1.xml
ADDED
|
@@ -0,0 +1,413 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<mujoco model="g1">
|
| 2 |
+
<compiler angle="radian" meshdir="../meshes/g1"/>
|
| 3 |
+
|
| 4 |
+
<default>
|
| 5 |
+
<default class="g1">
|
| 6 |
+
<geom contype="0" conaffinity="0"/>
|
| 7 |
+
|
| 8 |
+
<joint frictionloss="0.1" solimplimit="0.97 0.995 0.001"/>
|
| 9 |
+
|
| 10 |
+
<default class="hip">
|
| 11 |
+
<default class="hip_pitch">
|
| 12 |
+
<joint axis="0 1 0" range="-2.5307 2.8798" actuatorfrcrange="-88 88" armature="0.01017752004"/>
|
| 13 |
+
</default>
|
| 14 |
+
<default class="hip_roll">
|
| 15 |
+
<joint axis="1 0 0" actuatorfrcrange="-139 139" armature="0.025101925"/>
|
| 16 |
+
</default>
|
| 17 |
+
<default class="hip_yaw">
|
| 18 |
+
<joint axis="0 0 1" range="-2.7576 2.7576" actuatorfrcrange="-88 88" armature="0.01017752004"/>
|
| 19 |
+
</default>
|
| 20 |
+
</default>
|
| 21 |
+
<default class="knee">
|
| 22 |
+
<joint axis="0 1 0" range="-0.087267 2.8798" actuatorfrcrange="-139 139" armature="0.025101925"/>
|
| 23 |
+
</default>
|
| 24 |
+
<default class="ankle">
|
| 25 |
+
<default class="ankle_pitch">
|
| 26 |
+
<joint axis="0 1 0" range="-0.87267 0.5236" actuatorfrcrange="-50 50" armature="0.00721945"/>
|
| 27 |
+
</default>
|
| 28 |
+
<default class="ankle_roll">
|
| 29 |
+
<joint axis="1 0 0" range="-0.2618 0.2618" actuatorfrcrange="-50 50" armature="0.00721945"/>
|
| 30 |
+
</default>
|
| 31 |
+
</default>
|
| 32 |
+
<default class="waist_yaw">
|
| 33 |
+
<joint axis="0 0 1" range="-2.618 2.618" actuatorfrcrange="-88 88" armature="0.01017752004"/>
|
| 34 |
+
</default>
|
| 35 |
+
<default class="waist_pitch">
|
| 36 |
+
<joint axis="0 1 0" range="-0.52 0.52" actuatorfrcrange="-50 50" armature="0.00721945"/>
|
| 37 |
+
</default>
|
| 38 |
+
<default class="waist_roll">
|
| 39 |
+
<joint axis="1 0 0" range="-0.52 0.52" actuatorfrcrange="-50 50" armature="0.00721945"/>
|
| 40 |
+
</default>
|
| 41 |
+
<default class="shoulder">
|
| 42 |
+
<default class="shoulder_pitch">
|
| 43 |
+
<joint axis="0 1 0" range="-3.0892 2.6704" actuatorfrcrange="-25 25" armature="0.003609725"/>
|
| 44 |
+
</default>
|
| 45 |
+
<default class="shoulder_roll">
|
| 46 |
+
<joint axis="1 0 0" actuatorfrcrange="-25 25" armature="0.003609725"/>
|
| 47 |
+
</default>
|
| 48 |
+
<default class="shoulder_yaw">
|
| 49 |
+
<joint axis="0 0 1" range="-2.618 2.618" actuatorfrcrange="-25 25" armature="0.003609725"/>
|
| 50 |
+
</default>
|
| 51 |
+
</default>
|
| 52 |
+
<default class="elbow">
|
| 53 |
+
<joint axis="0 1 0" range="-1.0472 2.0944" actuatorfrcrange="-25 25" armature="0.003609725"/>
|
| 54 |
+
</default>
|
| 55 |
+
<default class="wrist">
|
| 56 |
+
<default class="wrist_roll">
|
| 57 |
+
<joint axis="1 0 0" range="-1.97222 1.97222" actuatorfrcrange="-25 25" armature="0.003609725"/>
|
| 58 |
+
</default>
|
| 59 |
+
<default class="wrist_pitch">
|
| 60 |
+
<joint axis="0 1 0" range="-1.61443 1.61443" actuatorfrcrange="-5 5" armature="0.00425"/>
|
| 61 |
+
</default>
|
| 62 |
+
<default class="wrist_yaw">
|
| 63 |
+
<joint axis="0 0 1" range="-1.61443 1.61443" actuatorfrcrange="-5 5" armature="0.00425"/>
|
| 64 |
+
</default>
|
| 65 |
+
</default>
|
| 66 |
+
|
| 67 |
+
<default class="visual">
|
| 68 |
+
<geom group="2" type="mesh" density="0" material="silver"/>
|
| 69 |
+
</default>
|
| 70 |
+
<default class="collision">
|
| 71 |
+
<geom group="3" rgba=".2 .6 .2 .3" contype="1" conaffinity="1"/>
|
| 72 |
+
<default class="foot">
|
| 73 |
+
<geom size="0.085 0.03 0.005"/>
|
| 74 |
+
</default>
|
| 75 |
+
</default>
|
| 76 |
+
<site group="5" rgba="1 0 0 1"/>
|
| 77 |
+
</default>
|
| 78 |
+
</default>
|
| 79 |
+
|
| 80 |
+
<asset>
|
| 81 |
+
<material name="silver" rgba="0.7 0.7 0.7 1"/>
|
| 82 |
+
<material name="black" rgba="0.2 0.2 0.2 1"/>
|
| 83 |
+
|
| 84 |
+
<mesh name="pelvis" file="pelvis.STL"/>
|
| 85 |
+
<mesh name="pelvis_contour_link" file="pelvis_contour_link.STL"/>
|
| 86 |
+
<mesh name="left_hip_pitch_link" file="left_hip_pitch_link.STL"/>
|
| 87 |
+
<mesh name="left_hip_roll_link" file="left_hip_roll_link.STL"/>
|
| 88 |
+
<mesh name="left_hip_yaw_link" file="left_hip_yaw_link.STL"/>
|
| 89 |
+
<mesh name="left_knee_link" file="left_knee_link.STL"/>
|
| 90 |
+
<mesh name="left_ankle_pitch_link"
|
| 91 |
+
file="left_ankle_pitch_link.STL"/>
|
| 92 |
+
<mesh name="left_ankle_roll_link" file="left_ankle_roll_link.STL"/>
|
| 93 |
+
<mesh name="right_hip_pitch_link" file="right_hip_pitch_link.STL"/>
|
| 94 |
+
<mesh name="right_hip_roll_link" file="right_hip_roll_link.STL"/>
|
| 95 |
+
<mesh name="right_hip_yaw_link" file="right_hip_yaw_link.STL"/>
|
| 96 |
+
<mesh name="right_knee_link" file="right_knee_link.STL"/>
|
| 97 |
+
<mesh name="right_ankle_pitch_link"
|
| 98 |
+
file="right_ankle_pitch_link.STL"/>
|
| 99 |
+
<mesh name="right_ankle_roll_link"
|
| 100 |
+
file="right_ankle_roll_link.STL"/>
|
| 101 |
+
<mesh name="waist_yaw_link" file="waist_yaw_link_rev_1_0.STL"/>
|
| 102 |
+
<mesh name="waist_roll_link" file="waist_roll_link_rev_1_0.STL"/>
|
| 103 |
+
<mesh name="torso_link" file="torso_link_rev_1_0.STL"/>
|
| 104 |
+
<mesh name="logo_link" file="logo_link.STL"/>
|
| 105 |
+
<mesh name="head_link" file="head_link.STL"/>
|
| 106 |
+
<mesh name="left_shoulder_pitch_link"
|
| 107 |
+
file="left_shoulder_pitch_link.STL"/>
|
| 108 |
+
<mesh name="left_shoulder_roll_link"
|
| 109 |
+
file="left_shoulder_roll_link.STL"/>
|
| 110 |
+
<mesh name="left_shoulder_yaw_link"
|
| 111 |
+
file="left_shoulder_yaw_link.STL"/>
|
| 112 |
+
<mesh name="left_elbow_link" file="left_elbow_link.STL"/>
|
| 113 |
+
<mesh name="left_wrist_roll_link" file="left_wrist_roll_link.STL"/>
|
| 114 |
+
<mesh name="left_wrist_pitch_link"
|
| 115 |
+
file="left_wrist_pitch_link.STL"/>
|
| 116 |
+
<mesh name="left_wrist_yaw_link" file="left_wrist_yaw_link.STL"/>
|
| 117 |
+
<mesh name="left_rubber_hand" file="left_rubber_hand.STL"/>
|
| 118 |
+
<mesh name="right_shoulder_pitch_link"
|
| 119 |
+
file="right_shoulder_pitch_link.STL"/>
|
| 120 |
+
<mesh name="right_shoulder_roll_link"
|
| 121 |
+
file="right_shoulder_roll_link.STL"/>
|
| 122 |
+
<mesh name="right_shoulder_yaw_link"
|
| 123 |
+
file="right_shoulder_yaw_link.STL"/>
|
| 124 |
+
<mesh name="right_elbow_link" file="right_elbow_link.STL"/>
|
| 125 |
+
<mesh name="right_wrist_roll_link"
|
| 126 |
+
file="right_wrist_roll_link.STL"/>
|
| 127 |
+
<mesh name="right_wrist_pitch_link"
|
| 128 |
+
file="right_wrist_pitch_link.STL"/>
|
| 129 |
+
<mesh name="right_wrist_yaw_link" file="right_wrist_yaw_link.STL"/>
|
| 130 |
+
<mesh name="right_rubber_hand" file="right_rubber_hand.STL"/>
|
| 131 |
+
</asset>
|
| 132 |
+
|
| 133 |
+
<worldbody>
|
| 134 |
+
<body name="pelvis" pos="0 0 0.793" childclass="g1">
|
| 135 |
+
<inertial pos="0 0 -0.07605" quat="1 0 -0.000399148 0" mass="3.813" diaginertia="0.010549 0.0093089 0.0079184"/>
|
| 136 |
+
<freejoint name="floating_base_joint"/>
|
| 137 |
+
<geom class="visual" material="black" mesh="pelvis"/>
|
| 138 |
+
<geom class="visual" mesh="pelvis_contour_link"/>
|
| 139 |
+
<geom mesh="pelvis_contour_link" class="visual"/>
|
| 140 |
+
<geom name="pelvis_collision" class="collision" type="sphere" size="0.07" pos="0 0 -0.08"/>
|
| 141 |
+
<site name="imu_in_pelvis" size="0.01" pos="0.04525 0 -0.08339"/>
|
| 142 |
+
<site name="pelvis" size="0.01" pos="0 0 0"/>
|
| 143 |
+
<body name="left_hip_pitch_link" pos="0 0.064452 -0.1027">
|
| 144 |
+
<inertial pos="0.002741 0.047791 -0.02606" quat="0.954862 0.293964 0.0302556 0.030122" mass="1.35"
|
| 145 |
+
diaginertia="0.00181517 0.00153422 0.00116212"/>
|
| 146 |
+
<joint name="left_hip_pitch_joint" class="hip_pitch"/>
|
| 147 |
+
<geom class="visual" material="black" mesh="left_hip_pitch_link"/>
|
| 148 |
+
<geom material="black" mesh="left_hip_pitch_link" class="visual"/>
|
| 149 |
+
<body name="left_hip_roll_link" pos="0 0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
|
| 150 |
+
<inertial pos="0.029812 -0.001045 -0.087934" quat="0.977808 -1.97119e-05 0.205576 -0.0403793" mass="1.52"
|
| 151 |
+
diaginertia="0.00254986 0.00241169 0.00148755"/>
|
| 152 |
+
<joint name="left_hip_roll_joint" class="hip_roll" range="-0.5236 2.9671"/>
|
| 153 |
+
<geom class="visual" mesh="left_hip_roll_link"/>
|
| 154 |
+
<geom mesh="left_hip_roll_link" class="visual"/>
|
| 155 |
+
<geom name="left_thigh" class="collision" type="capsule" size="0.05" fromto="0.02 0 0 0.02 0 -0.2"/>
|
| 156 |
+
<body name="left_hip_yaw_link" pos="0.025001 0 -0.12412">
|
| 157 |
+
<inertial pos="-0.057709 -0.010981 -0.15078" quat="0.600598 0.15832 0.223482 0.751181" mass="1.702"
|
| 158 |
+
diaginertia="0.00776166 0.00717575 0.00160139"/>
|
| 159 |
+
<joint name="left_hip_yaw_joint" class="hip_yaw"/>
|
| 160 |
+
<geom class="visual" mesh="left_hip_yaw_link"/>
|
| 161 |
+
<body name="left_knee_link" pos="-0.078273 0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
|
| 162 |
+
<inertial pos="0.005457 0.003964 -0.12074" quat="0.923418 -0.0327699 0.0158246 0.382067" mass="1.932"
|
| 163 |
+
diaginertia="0.0113804 0.0112778 0.00146458"/>
|
| 164 |
+
<joint name="left_knee_joint" class="knee"/>
|
| 165 |
+
<geom class="visual" mesh="left_knee_link"/>
|
| 166 |
+
<geom name="left_shin" class="collision" type="capsule" size="0.04" fromto="0.02 0 0 0.02 0 -0.25"/>
|
| 167 |
+
<body name="left_ankle_pitch_link" pos="0 -9.4445e-05 -0.30001">
|
| 168 |
+
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074"
|
| 169 |
+
diaginertia="1.89e-05 1.40805e-05 6.9195e-06"/>
|
| 170 |
+
<joint name="left_ankle_pitch_joint" class="ankle_pitch"/>
|
| 171 |
+
<geom class="visual" mesh="left_ankle_pitch_link"/>
|
| 172 |
+
<body name="left_ankle_roll_link" pos="0 0 -0.017558">
|
| 173 |
+
<site name="left_foot" rgba="1 0 0 1"/>
|
| 174 |
+
<inertial pos="0.026505 0 -0.016425" quat="-0.000481092 0.728482 -0.000618967 0.685065" mass="0.608"
|
| 175 |
+
diaginertia="0.00167218 0.0016161 0.000217621"/>
|
| 176 |
+
<joint name="left_ankle_roll_joint" class="ankle_roll"/>
|
| 177 |
+
<geom class="visual" material="black" mesh="left_ankle_roll_link"/>
|
| 178 |
+
<geom name="left_foot1_collision" type="capsule" size="0.01" class="collision" fromto="0.1 -0.026 -0.025 0.05 -0.027 -0.025"/>
|
| 179 |
+
<geom name="left_foot2_collision" type="capsule" size="0.01" class="collision" fromto="-0.044 -0.018 -0.025 0.123 -0.018 -0.025"/>
|
| 180 |
+
<geom name="left_foot3_collision" type="capsule" size="0.01" class="collision" fromto="-0.052 -0.01 -0.025 0.13 -0.01 -0.025"/>
|
| 181 |
+
<geom name="left_foot4_collision" type="capsule" size="0.01" class="collision" fromto="-0.054 0 -0.025 0.132 0 -0.025"/>
|
| 182 |
+
<geom name="left_foot5_collision" type="capsule" size="0.01" class="collision" fromto="-0.052 0.01 -0.025 0.13 0.01 -0.025"/>
|
| 183 |
+
<geom name="left_foot6_collision" type="capsule" size="0.01" class="collision" fromto="-0.044 0.018 -0.025 0.123 0.018 -0.025"/>
|
| 184 |
+
<geom name="left_foot7_collision" type="capsule" size="0.01" class="collision" fromto="0.1 0.026 -0.025 0.05 0.026 -0.025"/>
|
| 185 |
+
</body>
|
| 186 |
+
</body>
|
| 187 |
+
</body>
|
| 188 |
+
</body>
|
| 189 |
+
</body>
|
| 190 |
+
</body>
|
| 191 |
+
<body name="right_hip_pitch_link" pos="0 -0.064452 -0.1027">
|
| 192 |
+
<inertial pos="0.002741 -0.047791 -0.02606" quat="0.954862 -0.293964 0.0302556 -0.030122" mass="1.35"
|
| 193 |
+
diaginertia="0.00181517 0.00153422 0.00116212"/>
|
| 194 |
+
<joint name="right_hip_pitch_joint" class="hip_pitch"/>
|
| 195 |
+
<geom class="visual" material="black" mesh="right_hip_pitch_link"/>
|
| 196 |
+
<body name="right_hip_roll_link" pos="0 -0.052 -0.030465" quat="0.996179 0 -0.0873386 0">
|
| 197 |
+
<inertial pos="0.029812 0.001045 -0.087934" quat="0.977808 1.97119e-05 0.205576 0.0403793" mass="1.52"
|
| 198 |
+
diaginertia="0.00254986 0.00241169 0.00148755"/>
|
| 199 |
+
<joint name="right_hip_roll_joint" class="hip_roll" range="-2.9671 0.5236"/>
|
| 200 |
+
<geom class="visual" mesh="right_hip_roll_link"/>
|
| 201 |
+
<geom name="right_thigh" class="collision" type="capsule" size="0.05" fromto="0.02 0 0 0.02 0 -0.2"/>
|
| 202 |
+
<body name="right_hip_yaw_link" pos="0.025001 0 -0.12412">
|
| 203 |
+
<inertial pos="-0.057709 0.010981 -0.15078" quat="0.751181 0.223482 0.15832 0.600598" mass="1.702"
|
| 204 |
+
diaginertia="0.00776166 0.00717575 0.00160139"/>
|
| 205 |
+
<joint name="right_hip_yaw_joint" class="hip_yaw"/>
|
| 206 |
+
<geom class="visual" mesh="right_hip_yaw_link"/>
|
| 207 |
+
<body name="right_knee_link" pos="-0.078273 -0.0021489 -0.17734" quat="0.996179 0 0.0873386 0">
|
| 208 |
+
<inertial pos="0.005457 -0.003964 -0.12074" quat="0.923439 0.0345276 0.0116333 -0.382012" mass="1.932"
|
| 209 |
+
diaginertia="0.011374 0.0112843 0.00146452"/>
|
| 210 |
+
<joint name="right_knee_joint" class="knee"/>
|
| 211 |
+
<geom class="visual" mesh="right_knee_link"/>
|
| 212 |
+
<geom name="right_shin" class="collision" type="capsule" size="0.04" fromto="0.02 0 0 0.02 0 -0.25"/>
|
| 213 |
+
<body name="right_ankle_pitch_link" pos="0 9.4445e-05 -0.30001">
|
| 214 |
+
<inertial pos="-0.007269 0 0.011137" quat="0.603053 0.369225 0.369225 0.603053" mass="0.074"
|
| 215 |
+
diaginertia="1.89e-05 1.40805e-05 6.9195e-06"/>
|
| 216 |
+
<joint name="right_ankle_pitch_joint" class="ankle_pitch"/>
|
| 217 |
+
<geom class="visual" mesh="right_ankle_pitch_link"/>
|
| 218 |
+
<body name="right_ankle_roll_link" pos="0 0 -0.017558">
|
| 219 |
+
<site name="right_foot" pos="0 0 0"/>
|
| 220 |
+
<inertial pos="0.026505 0 -0.016425" quat="0.000481092 0.728482 0.000618967 0.685065" mass="0.608"
|
| 221 |
+
diaginertia="0.00167218 0.0016161 0.000217621"/>
|
| 222 |
+
<joint name="right_ankle_roll_joint" class="ankle_roll"/>
|
| 223 |
+
<geom class="visual" material="black" mesh="right_ankle_roll_link"/>
|
| 224 |
+
<geom name="right_foot1_collision" type="capsule" size="0.01" class="collision" fromto="0.1 -0.026 -0.025 0.05 -0.026 -0.025"/>
|
| 225 |
+
<geom name="right_foot2_collision" type="capsule" size="0.008" class="collision" fromto="-0.044 -0.018 -0.025 0.123 -0.018 -0.025"/>
|
| 226 |
+
<geom name="right_foot3_collision" type="capsule" size="0.01" class="collision" fromto="-0.052 -0.01 -0.025 0.13 -0.01 -0.025"/>
|
| 227 |
+
<geom name="right_foot4_collision" type="capsule" size="0.01" class="collision" fromto="-0.054 0 -0.025 0.132 0 -0.025"/>
|
| 228 |
+
<geom name="right_foot5_collision" type="capsule" size="0.01" class="collision" fromto="-0.052 0.01 -0.025 0.13 0.01 -0.025"/>
|
| 229 |
+
<geom name="right_foot6_collision" type="capsule" size="0.008" class="collision" fromto="-0.044 0.018 -0.025 0.123 0.018 -0.025"/>
|
| 230 |
+
<geom name="right_foot7_collision" type="capsule" size="0.01" class="collision" fromto="0.1 0.026 -0.025 0.05 0.026 -0.025"/>
|
| 231 |
+
</body>
|
| 232 |
+
</body>
|
| 233 |
+
</body>
|
| 234 |
+
</body>
|
| 235 |
+
</body>
|
| 236 |
+
</body>
|
| 237 |
+
<body name="waist_yaw_link">
|
| 238 |
+
<inertial pos="0.003494 0.000233 0.018034" quat="0.289697 0.591001 -0.337795 0.672821" mass="0.214"
|
| 239 |
+
diaginertia="0.000163531 0.000107714 0.000102205"/>
|
| 240 |
+
<joint name="waist_yaw_joint" class="waist_yaw"/>
|
| 241 |
+
<geom class="visual" mesh="waist_yaw_link"/>
|
| 242 |
+
<body name="waist_roll_link" pos="-0.0039635 0 0.044">
|
| 243 |
+
<inertial pos="0 2.3e-05 0" quat="0.5 0.5 -0.5 0.5" mass="0.086" diaginertia="8.245e-06 7.079e-06 6.339e-06"/>
|
| 244 |
+
<joint name="waist_roll_joint" class="waist_roll"/>
|
| 245 |
+
<geom class="visual" mesh="waist_roll_link"/>
|
| 246 |
+
<body name="torso_link">
|
| 247 |
+
<inertial pos="0.00203158 0.000339683 0.184568" quat="0.999803 -6.03319e-05 0.0198256 0.00131986"
|
| 248 |
+
mass="7.818" diaginertia="0.121847 0.109825 0.0273735"/>
|
| 249 |
+
<joint name="waist_pitch_joint" class="waist_pitch"/>
|
| 250 |
+
<geom class="visual" mesh="torso_link"/>
|
| 251 |
+
<geom pos="0.0039635 0 -0.044" quat="1 0 0 0" class="visual" material="black" mesh="logo_link"/>
|
| 252 |
+
<geom pos="0.0039635 0 -0.044" class="visual" material="black" mesh="head_link"/>
|
| 253 |
+
|
| 254 |
+
<geom name="torso_collision1" class="collision" type="capsule" size="0.073" fromto="0.005 -0.032 .22 0.005 0.032 .22"/>
|
| 255 |
+
<geom name="torso_collision2" class="collision" type="capsule" size="0.07" fromto="0.005 -0.028 .13 0.005 0.028 .13"/>
|
| 256 |
+
<geom name="torso_collision3" class="collision" type="capsule" size="0.065" fromto="0.005 -0.02 .06 0.005 0.02 .06"/>
|
| 257 |
+
<geom name="head_collision" class="collision" type="capsule" size="0.068" fromto="0.01 0 .41 0.01 0 .42"/>
|
| 258 |
+
|
| 259 |
+
<site name="imu_in_torso" size="0.01" pos="-0.03959 -0.00224 0.14792"/>
|
| 260 |
+
<site name="mid360" size="0.01" pos="0.0002835 0.00003 0.41618" quat="0.00000094 -0.99979404 0.00004632 0.02029493"/>
|
| 261 |
+
<body name="left_shoulder_pitch_link" pos="0.0039563 0.10022 0.24778"
|
| 262 |
+
quat="0.990264 0.139201 1.38722e-05 -9.86868e-05">
|
| 263 |
+
<inertial pos="0 0.035892 -0.011628" quat="0.654152 0.0130458 -0.326267 0.68225" mass="0.718"
|
| 264 |
+
diaginertia="0.000465864 0.000432842 0.000406394"/>
|
| 265 |
+
<joint name="left_shoulder_pitch_joint" class="shoulder_pitch"/>
|
| 266 |
+
<geom class="visual" mesh="left_shoulder_pitch_link"/>
|
| 267 |
+
<geom size="0.03 0.025" pos="0 0.04 -0.01" quat="0.707107 0 0.707107 0" type="cylinder" class="visual"/>
|
| 268 |
+
<body name="left_shoulder_roll_link" pos="0 0.038 -0.013831" quat="0.990268 -0.139172 0 0">
|
| 269 |
+
<inertial pos="-0.000227 0.00727 -0.063243" quat="0.701256 -0.0196223 -0.00710317 0.712604" mass="0.643"
|
| 270 |
+
diaginertia="0.000691311 0.000618011 0.000388977"/>
|
| 271 |
+
<joint name="left_shoulder_roll_joint" range="-1.5882 2.2515" class="shoulder_roll"/>
|
| 272 |
+
<geom class="visual" mesh="left_shoulder_roll_link"/>
|
| 273 |
+
<geom size="0.03 0.015" pos="-0.004 0.006 -0.053" type="cylinder" class="visual"/>
|
| 274 |
+
<body name="left_shoulder_yaw_link" pos="0 0.00624 -0.1032">
|
| 275 |
+
<inertial pos="0.010773 -0.002949 -0.072009" quat="0.716879 -0.0964829 -0.0679942 0.687134"
|
| 276 |
+
mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661"/>
|
| 277 |
+
<joint name="left_shoulder_yaw_joint" class="shoulder_yaw"/>
|
| 278 |
+
<geom class="visual" mesh="left_shoulder_yaw_link"/>
|
| 279 |
+
<geom name="left_shoulder_yaw_collision" class="collision" type="capsule" size="0.035" fromto="0 0 -0.08 0 0 0.05"/>
|
| 280 |
+
<body name="left_elbow_link" pos="0.015783 0 -0.080518">
|
| 281 |
+
<inertial pos="0.064956 0.004454 -0.010062" quat="0.541765 0.636132 0.388821 0.388129" mass="0.6"
|
| 282 |
+
diaginertia="0.000443035 0.000421612 0.000259353"/>
|
| 283 |
+
<joint name="left_elbow_joint" class="elbow"/>
|
| 284 |
+
<geom class="visual" mesh="left_elbow_link"/>
|
| 285 |
+
<geom name="left_elbow_yaw_collision" class="collision" type="capsule" size="0.035" fromto="-0.01 0 -0.01 0.12 0 -0.01"/>
|
| 286 |
+
<body name="left_wrist_roll_link" pos="0.1 0.00188791 -0.01">
|
| 287 |
+
<inertial pos="0.0171394 0.000537591 4.8864e-07" quat="0.575338 0.411667 -0.574906 0.411094"
|
| 288 |
+
mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05"/>
|
| 289 |
+
<joint name="left_wrist_roll_joint" class="wrist_roll"/>
|
| 290 |
+
<geom class="visual" mesh="left_wrist_roll_link"/>
|
| 291 |
+
<body name="left_wrist_pitch_link" pos="0.038 0 0">
|
| 292 |
+
<inertial pos="0.0229999 -0.00111685 -0.00111658" quat="0.249998 0.661363 0.293036 0.643608"
|
| 293 |
+
mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648"/>
|
| 294 |
+
<joint name="left_wrist_pitch_joint" class="wrist_pitch"/>
|
| 295 |
+
<geom class="visual" mesh="left_wrist_pitch_link"/>
|
| 296 |
+
<body name="left_wrist_yaw_link" pos="0.046 0 0">
|
| 297 |
+
<inertial pos="0.0708244 0.000191745 0.00161742" quat="0.510571 0.526295 0.468078 0.493188"
|
| 298 |
+
mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566"/>
|
| 299 |
+
<joint name="left_wrist_yaw_joint" class="wrist_yaw"/>
|
| 300 |
+
<geom class="visual" mesh="left_wrist_yaw_link"/>
|
| 301 |
+
<geom pos="0.0415 0.003 0" quat="1 0 0 0" class="visual" mesh="left_rubber_hand"/>
|
| 302 |
+
<site name="left_palm" pos="0.08 0 0" size="0.01"/>
|
| 303 |
+
<geom name="left_hand_collision" class="collision" type="capsule" size="0.05" fromto="0.05 0 0 0.1 0 0"/>
|
| 304 |
+
</body>
|
| 305 |
+
</body>
|
| 306 |
+
</body>
|
| 307 |
+
</body>
|
| 308 |
+
</body>
|
| 309 |
+
</body>
|
| 310 |
+
</body>
|
| 311 |
+
<body name="right_shoulder_pitch_link" pos="0.0039563 -0.10021 0.24778"
|
| 312 |
+
quat="0.990264 -0.139201 1.38722e-05 9.86868e-05">
|
| 313 |
+
<inertial pos="0 -0.035892 -0.011628" quat="0.68225 -0.326267 0.0130458 0.654152" mass="0.718"
|
| 314 |
+
diaginertia="0.000465864 0.000432842 0.000406394"/>
|
| 315 |
+
<joint name="right_shoulder_pitch_joint" class="shoulder_pitch"/>
|
| 316 |
+
<geom class="visual" mesh="right_shoulder_pitch_link"/>
|
| 317 |
+
<geom size="0.03 0.025" pos="0 -0.04 -0.01" quat="0.707107 0 0.707107 0" type="cylinder" class="visual"/>
|
| 318 |
+
<body name="right_shoulder_roll_link" pos="0 -0.038 -0.013831" quat="0.990268 0.139172 0 0">
|
| 319 |
+
<inertial pos="-0.000227 -0.00727 -0.063243" quat="0.712604 -0.00710317 -0.0196223 0.701256"
|
| 320 |
+
mass="0.643" diaginertia="0.000691311 0.000618011 0.000388977"/>
|
| 321 |
+
<joint name="right_shoulder_roll_joint" range="-2.2515 1.5882" class="shoulder_roll"/>
|
| 322 |
+
<geom class="visual" mesh="right_shoulder_roll_link"/>
|
| 323 |
+
<geom size="0.03 0.015" pos="-0.004 -0.006 -0.053" type="cylinder" class="visual"/>
|
| 324 |
+
<body name="right_shoulder_yaw_link" pos="0 -0.00624 -0.1032">
|
| 325 |
+
<inertial pos="0.010773 0.002949 -0.072009" quat="0.687134 -0.0679942 -0.0964829 0.716879"
|
| 326 |
+
mass="0.734" diaginertia="0.00106187 0.00103217 0.000400661"/>
|
| 327 |
+
<joint name="right_shoulder_yaw_joint" class="shoulder_yaw"/>
|
| 328 |
+
<geom class="visual" mesh="right_shoulder_yaw_link"/>
|
| 329 |
+
<geom name="right_shoulder_yaw_collision" class="collision" type="capsule" size="0.035" fromto="0 0 -0.08 0 0 0.05"/>
|
| 330 |
+
<body name="right_elbow_link" pos="0.015783 0 -0.080518">
|
| 331 |
+
<inertial pos="0.064956 -0.004454 -0.010062" quat="0.388129 0.388821 0.636132 0.541765" mass="0.6"
|
| 332 |
+
diaginertia="0.000443035 0.000421612 0.000259353"/>
|
| 333 |
+
<joint name="right_elbow_joint" class="elbow"/>
|
| 334 |
+
<geom class="visual" mesh="right_elbow_link"/>
|
| 335 |
+
<geom name="right_elbow_yaw_collision" class="collision" type="capsule" size="0.035" fromto="-0.01 0 -0.01 0.12 0 -0.01"/>
|
| 336 |
+
<body name="right_wrist_roll_link" pos="0.1 -0.00188791 -0.01">
|
| 337 |
+
<inertial pos="0.0171394 -0.000537591 4.8864e-07" quat="0.411667 0.575338 -0.411094 0.574906"
|
| 338 |
+
mass="0.085445" diaginertia="5.48211e-05 4.96646e-05 3.57798e-05"/>
|
| 339 |
+
<joint name="right_wrist_roll_joint" class="wrist_roll"/>
|
| 340 |
+
<geom class="visual" mesh="right_wrist_roll_link"/>
|
| 341 |
+
<body name="right_wrist_pitch_link" pos="0.038 0 0">
|
| 342 |
+
<inertial pos="0.0229999 0.00111685 -0.00111658" quat="0.643608 0.293036 0.661363 0.249998"
|
| 343 |
+
mass="0.48405" diaginertia="0.000430353 0.000429873 0.000164648"/>
|
| 344 |
+
<joint name="right_wrist_pitch_joint" class="wrist_pitch"/>
|
| 345 |
+
<geom class="visual" mesh="right_wrist_pitch_link"/>
|
| 346 |
+
<body name="right_wrist_yaw_link" pos="0.046 0 0">
|
| 347 |
+
<inertial pos="0.0708244 -0.000191745 0.00161742" quat="0.493188 0.468078 0.526295 0.510571"
|
| 348 |
+
mass="0.254576" diaginertia="0.000646113 0.000559993 0.000147566"/>
|
| 349 |
+
<joint name="right_wrist_yaw_joint" class="wrist_yaw"/>
|
| 350 |
+
<geom class="visual" mesh="right_wrist_yaw_link"/>
|
| 351 |
+
<geom pos="0.0415 -0.003 0" quat="1 0 0 0" class="visual" mesh="right_rubber_hand"/>
|
| 352 |
+
<site name="right_palm" pos="0.08 0 0" size="0.01"/>
|
| 353 |
+
<geom name="right_hand_collision" class="collision" type="capsule" size="0.05" fromto="0.05 0 0 0.1 0 0"/>
|
| 354 |
+
</body>
|
| 355 |
+
</body>
|
| 356 |
+
</body>
|
| 357 |
+
</body>
|
| 358 |
+
</body>
|
| 359 |
+
</body>
|
| 360 |
+
</body>
|
| 361 |
+
</body>
|
| 362 |
+
</body>
|
| 363 |
+
</body>
|
| 364 |
+
</body>
|
| 365 |
+
</worldbody>
|
| 366 |
+
|
| 367 |
+
<contact>>
|
| 368 |
+
<!-- left foot - floor -->
|
| 369 |
+
<pair name="left_foot1_floor" geom1="left_foot1_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 370 |
+
<pair name="left_foot2_floor" geom1="left_foot2_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 371 |
+
<pair name="left_foot3_floor" geom1="left_foot3_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 372 |
+
<pair name="left_foot4_floor" geom1="left_foot4_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 373 |
+
<pair name="left_foot5_floor" geom1="left_foot5_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 374 |
+
<pair name="left_foot6_floor" geom1="left_foot6_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 375 |
+
<pair name="left_foot7_floor" geom1="left_foot7_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 376 |
+
<!-- right foot - floor -->
|
| 377 |
+
<pair name="right_foot1_floor" geom1="right_foot1_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 378 |
+
<pair name="right_foot2_floor" geom1="right_foot2_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 379 |
+
<pair name="right_foot3_floor" geom1="right_foot3_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 380 |
+
<pair name="right_foot4_floor" geom1="right_foot4_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 381 |
+
<pair name="right_foot5_floor" geom1="right_foot5_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 382 |
+
<pair name="right_foot6_floor" geom1="right_foot6_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 383 |
+
<pair name="right_foot7_floor" geom1="right_foot7_collision" geom2="floor" solref="0.01 1" friction="0.8 0.8"/>
|
| 384 |
+
</contact>
|
| 385 |
+
|
| 386 |
+
<sensor>
|
| 387 |
+
<framequat name="base_quat" objtype="site" objname="imu_in_pelvis"/>
|
| 388 |
+
<gyro name="base_gyro" site="imu_in_pelvis"/>
|
| 389 |
+
<accelerometer name="base_accel" site="imu_in_pelvis"/>
|
| 390 |
+
|
| 391 |
+
<framequat name="mid360_quat" objtype="site" objname="mid360"/>
|
| 392 |
+
<framepos name="mid360_pos" objtype="site" objname="mid360"/>
|
| 393 |
+
</sensor>
|
| 394 |
+
|
| 395 |
+
<visual>
|
| 396 |
+
<headlight diffuse="0.6 0.6 0.6" ambient="0.3 0.3 0.3" specular="0 0 0"/>
|
| 397 |
+
<rgba haze="0.15 0.25 0.35 1"/>
|
| 398 |
+
<global azimuth="120" elevation="-20"/>
|
| 399 |
+
</visual>
|
| 400 |
+
|
| 401 |
+
<asset>
|
| 402 |
+
<texture type="skybox" builtin="gradient" rgb1="0.3 0.5 0.7" rgb2="0 0 0" width="512" height="3072"/>
|
| 403 |
+
<texture type="2d" name="groundplane" builtin="checker" mark="edge" rgb1="0.2 0.3 0.4" rgb2="0.1 0.2 0.3"
|
| 404 |
+
markrgb="0.8 0.8 0.8" width="300" height="300"/>
|
| 405 |
+
<material name="groundplane" texture="groundplane" texuniform="true" texrepeat="5 5" reflectance="0.2"/>
|
| 406 |
+
</asset>
|
| 407 |
+
|
| 408 |
+
<worldbody>
|
| 409 |
+
<light pos="0 0 1.5" dir="0 0 -1" directional="true"/>
|
| 410 |
+
<geom name="floor" size="0 0 0.01" type="plane" material="groundplane" conaffinity="1"/>
|
| 411 |
+
</worldbody>
|
| 412 |
+
|
| 413 |
+
</mujoco>
|
ardy/assets/skeletons/somaskel30/joints.p
ADDED
|
Binary file (1.88 kB). View file
|
|
|