Bilingual

Segmenting Robot Video into Actionable Subtasks
6 days ago
- Introducing WGO-Bench, a benchmark for robotics subtask annotation across 100 video episodes with 743 segments covering 62 task instructions.
- Over 60 experiments identified the best subtask annotation pipeline: best segmentation F1 of 0.306, labeling accuracy of 61.0%, and end-to-end F1 of 0.168.
- Gemini models, particularly Gemini 3.5 Flash, outperformed other models by 24.5% and are best for this task.
- Cost-effective method uses contact sheets, costing $2.64 per hour of video (batch pricing), roughly 19x cheaper than human annotation.
- Key techniques include visual timestamping on contact sheets, strict annotation protocols, and using previous/current/next segment context for labeling.
- Segmentation is the main bottleneck, especially for short subtasks, while labeling improves with contextual visual input.
- Open-sourced pipeline in Refiner for public use, enabling scalable subtask annotation without human intervention.
AI is 'not smart' so what's next in artificial intelligence?
3 days ago
- Yann LeCun critiques current LLMs (like ChatGPT) for lacking real-world understanding and physical reasoning, comparing them unfavorably to even animals like rats.
- He left Meta to found AMI Labs, aiming to develop AI beyond LLMs, focusing on flexible systems for complex real-world tasks such as robotics and household chores.
- AMI Labs raised over $1bn in seed funding, with backers including Nvidia and Jeff Bezos's fund, highlighting investor confidence in next-generation AI.
- LeCun's approach, Joint Embedding Predictive Architecture (JEPA), creates abstractions to filter useless information and predict outcomes without over-predicting, like not guessing a pen's fall direction.
- Other researchers, like Ingmar Posner at Oxford, are working on 'World Models' that simulate environments to enable causal reasoning and explainable AI, inspired by advances since 2018.
- Projects like Google's Dreamer, DeepMind's Genie, and Wayve's Gaia exemplify the shift towards AI that learns from mental simulations, with applications in gaming and robotics.
- LeCun predicts AMI Labs' model will be tested in industrial settings soon, with long-term goals for general intelligence requiring minimal training.
- Despite AI advancements, humans will remain crucial for asking questions, creativity, and leadership, with AI acting as smarter assistants.
Heuristics for lab robotics, and where its future may go
2 days ago
- There are two main types of lab robots: 'boxes' (like liquid handlers) that automate specific tasks and 'arms' (like the UR5) that move materials between boxes to form integrated workcells.
- Most lab protocols can be automated, but often aren't worth automating because the upfront investment in time and cost is only justified for high-throughput, repetitive tasks, not for exploratory research that changes frequently.
- Three approaches to improve lab automation are: better translation layers (e.g., Synthace, Briefly Bio) to convert human intent into robot instructions, better physical hardware integration (e.g., Automata, Ginkgo Bioworks) to connect systems, and adding intelligence (e.g., Medra, Zeon Systems) for error recovery and adaptability.
- The long-term business model for lab automation likely converges on 'cloud labs' (like the defunct Transcriptic or Emerald Cloud Labs), which centralize experiments to maximize robot utilization and can vertically integrate to reduce costs, though early attempts struggled by being too general and offloading logistics to users.
- The ultimate value of lab robotics may not be just increasing throughput, but enabling better, more predictive assays (improving 'predictive validity') and higher-standard experimentation at scale, potentially transforming drug discovery.
Drone Autonomy Crash Course
a day ago
- A short guide to drone autonomy focusing on modeling, state estimation, motion planning, and control for quadcopters.
- Aimed at newcomers to provide an intuitive overview and serve as a 'jumping off point' with references for deeper study.
- Based on the author's experience with the challenge of understanding academic papers on quadcopter autonomy.
- Originally a long paper but broken into digestible pieces with casual writing style for accessibility.
- Includes incomplete sections (WIP) to be updated soon, emphasizing practical understanding over rigor.

Hasty Briefsbeta

#robotics

Segmenting Robot Video into Actionable Subtasks

AI is 'not smart' so what's next in artificial intelligence?

Heuristics for lab robotics, and where its future may go

Drone Autonomy Crash Course