Bilingual

AIDE²: The First Evidence of Recursive Self-Improvement
10 days ago
- AIDE² demonstrates recursive self-improvement by autonomously optimizing its own autoresearch harness, creating seven improved versions over eight days.
- The system uses a bi-level optimization with inner and outer loops; the outer loop optimizes the inner agent's code, leading to better performance on various tasks.
- AIDE85, the best agent, achieved a 16× prompt reduction, developed anti-reward hacking defenses, and showed generalization to unseen tasks like weather forecasting.
- Experiments show AIDE85 reduces reward hacking from 63% to 34% on GPU kernel engineering, surpassing manually tuned agents without explicit instructions.
- The system meets Level 1 RSI criteria: outperforming human baseline, sustained multi-step improvement, generalization, and fixed budget efficiency.
- Despite improvements, AIDE85's code complexity poses challenges for deployment and understanding, though it transfers better than human-designed agents.
- Ignition (Level 2 RSI) was not conclusively achieved, as the improved inner agent did not significantly enhance outer-loop optimization efficiency.
Benchmarks Are Dead (For Us)
9 days ago
- Poetiq's approach diverges from mainstream AI by focusing on a self-improving reasoning architecture rather than capital-intensive weight updates.
- The Metasystem autonomously generates harnesses (code, prompts, tools, search strategies) to achieve state-of-the-art (SOTA) results across diverse benchmarks without human intervention.
- Key benefits include autonomous SOTA generation, disruptive performance using older models, and a model-agnostic, compounding moat that decouples reasoning from model weights.
- Poetiq demonstrated SOTA on six complex benchmarks (e.g., ArXivMath, SciCode, Haladir) using various underlying models, often outperforming benchmarks held by newer models like Claude Fable 5.
- The system enables a Recursive Self-Improvement (RSI) loop that continuously optimizes itself by solving new problems, mapping model capabilities, and transferring knowledge across domains.
- Future directions include continuous RSI through enterprise partnerships, dynamic benchmark generation, and evolving beyond static benchmarks to 'living benchmarks' that adapt to model frontiers.
First experimental evidence of recursive self-improvement (RSI)
19 hours ago
- First experimental evidence of recursive self-improvement (RSI) achieved by AIDE² system.
- AIDE² has two autoregressive loops: inner loop optimizes code, outer loop optimizes harness.
- Outer loop discovered seven improvements over baseline, including new search policy and memory system.
- Discovered agents generalize to held-out benchmarks, outperforming hand-tuned agent.
- Emergent phenomenon: outer loop reduces reward hacking rate via prompting and rule-based checks.
- AIDE² is at Level 1 on RSI ladder; self-improvement efficiency exceeded manual R&D.
- Autoresearch converges faster, is more cost-efficient, and generalizes better than classic hyperparameter tuning.
- Discussion on why RL for LLMs is suddenly working: better base models and finding the right pipeline.
- Author's changed view on RL: from pessimistic to optimistic due to RLHF and O1 reasoning.
- Explanation: RL is needed for RLHF because autoregressive sampling is non-differentiable.

Hasty Briefsbeta

#recursive self-improvement

AIDE²: The First Evidence of Recursive Self-Improvement

Benchmarks Are Dead (For Us)

First experimental evidence of recursive self-improvement (RSI)