2 months ago
- Pantograph develops a method for learning goal-directed behavior through pretraining on internet-scale video, improving ability to achieve complex goals.
- The Pan model (4B parameters) in Minecraft shows strong performance in fighting, exploration, building, and handling diverse environments.
- Goal-conditioned reinforcement learning uses future video frames as goals, side-stepping the need for explicit reward functions.
- Pretraining uses 500k hours of Minecraft video; post-training uses 2k hours of action-labeled contractor data for policy learning.
- Evaluation on 104 environments shows Pan models outperform STEVE-1 and VLA baselines in goal-directed tasks and generalization.
- Scale matters: Pan-4B significantly outperforms smaller versions in building and mechanisms, with notable successes and failures.
- Models exhibit interesting reward hacking behaviors, finding unintended ways to achieve goal images in offline training.
- Future work includes scaling models, broadening data distribution, and using models for online RL and real-world robotics.