Show HN: Kinematic Hand Skeleton Optimization in Jax
a year ago
- #motion-retargeting
- #Pi0
- #robotics
- A repository for training robots using Pi0 and Lerobot with human pose motion retargeting.
- Currently supports only Linux and requires the Pixi package manager.
- Commands to clone the repo and navigate into it.
- Instructions for obtaining camera intrinsics/extrinsics, depth maps, and point clouds from multiview images.
- Commands for camera calibration and dataset visualization for HOCap and Assembly101 datasets.
- Pipeline details for 2D pose estimation, tracking, and triangulation.
- Goal: Convert multi-camera footage into axis-angle joint angles and 3D positions for robotic retargeting.
- Benefits of using a kinematic skeleton with joint angles.
- Steps in the Capture-to-Angles Pipeline.
- Commands to run notebook tutorials for various functionalities.
- Future plans include adding egocentric views, pose retargeting to Isaac Sim, and fine-tuning on simulators.