Getting out of the way: my robotics crash course
2 days ago
- The author, a software engineer with 25 years of experience, built more software in the last 9 months using LLMs than in his entire career, creating 41 real projects.
- To make his work tangible for his kids, he bought a preassembled LeRobot SO-101 robotic arm kit with a webcam and used a Raspberry Pi 3 to control it.
- He used Codex (Astra) to set up the robot, including safety software for servos, calibration with AprilTags, and addressing power supply issues that limited servo movement.
- After using up Codex credits, he switched to Claude (Fable 5.1) and allowed it to work autonomously by sending a prompt to focus on gripping and placing blocks using camera feedback and resistance sensing.
- While at his daughter's volleyball practice, Claude independently tested mechanisms, built feedback loops, and successfully picked and placed blocks, despite initial aggressive movements that were self-corrected.
- The next day, he added more blocks (purple and blue) and had Claude place all three in a marked drop area, which it accomplished overnight, as shown in a timelapse video created by Codex.
- His kids were excited about the robot's performance, and the author plans to convert LLM training into VLA data for improved speed and cost, with a future goal of building a Jenga tower.