- FLUX 3 is a multimodal foundation model that generates audio-visual content and serves as the backbone for FLUX-mimic video-action models used in robotics.
- Video prediction is the most computationally intensive part, accounting for over 95% of training costs, as it forces the model to learn physical world dynamics like contact and motion.
- Adding action prediction to FLUX 3 initially degraded video quality by 10%, but after 3500 training steps, the model recovered full performance while also predicting actions.
- FLUX-mimic, built on FLUX 3, decodes actions from intermediate features of the video prediction path using a lightweight action decoder, achieving state-of-the-art success rates in robotic manipulation.
- The Self-Flow technique unifies generation and representation learning, improving both world model quality and downstream task performance for robotics.
- FLUX-mimic offers up to 10x sample efficiency over vision-language-action models and can run on a single NVIDIA RTX 5090 GPU with latency under 80ms for world representation.
- The model has been deployed in real factory settings at Audi, handling tasks like kitting, assembly, and handling soft materials that conventional automation couldn't manage.
- FLUX-mimic naturally recovers from task failures, such as missed grasps, without needing explicit demonstrations for every error scenario.