Systems optimization should be part of CI/CD

6 hours ago

LEVI is an AI-driven research framework for algorithmic discovery that achieves better results at a lower cost compared to existing frameworks like OpenEvolve and GEPA.
It uses a harness-first approach with stratified model allocation, assigning cheaper models (e.g., QWEN 30B) for most mutations and reserving expensive models for rare paradigm shifts.
LEVI maintains diversity in code structure and behavior using a fingerprint system and a CVT-MAP-Elites archive to prevent convergence and improve search efficiency.
Benchmark results show LEVI outperforms other frameworks, with an average score of 76.5, and reduces costs by 3–7 times, spending around $4.50 per problem versus $15–30 for baselines.
Controlled tests confirm LEVI's performance gains are due to its search architecture, not just model choice or budget, enabling efficient use of smaller models.
The framework supports continuous, bespoke optimization for real-world deployments, adapting algorithms to specific workloads, hardware, and SLOs as they change.
LEVI's Python API abstracts complexity, allowing users to focus on problem definition, and it is open-source with community engagement via Slack, Discord, and social media.
Lessons from using smaller models include managing higher error rates affordably, addressing reward hacking, prioritizing code over text, and balancing quantity versus evaluation time.
The ADRS initiative encourages collaboration and contributions to advance AI-driven research for systems optimization.

Hasty Briefsbeta