ai research

Bilingual

Great Question (YC W21) Is Hiring Senior Demand Gen Manager
a month ago
- Great Question is an AI-powered customer research platform for recruiting, running research, and sharing insights.
- The Demand Generation role owns SQL attainment and leads multi-channel strategy across organic, AI search, content, and paid acquisition.
- The role requires 5-8 years of B2B demand generation experience, AI-native practices, and strong channel judgment.
- Success in 12 months includes consistent SQL targets, repeatable programs, and AI-driven execution capacity.
- Company culture emphasizes high trust, high agency, emotional intelligence, and AI as a multiplier for creativity.
Lilian Weng joins OpenAI after saying Thinking Machine's pace hurt her health
2 months ago
- Lilian Weng, cofounder of Thinking Machines Lab, is returning to OpenAI to lead a team focused on accelerating internal AI research and achieving recursive self-improvement.
- She left Thinking Machines due to health concerns and the unsustainable pace required at the startup, seeking a more predictable role as an employee rather than a cofounder.
- Weng previously worked at OpenAI from 2018 to 2024, contributing to the development of GPT-4 and building the Applied AI Research team.
- Her departure from Thinking Machines is part of a trend, with nearly a third of founding members having left the startup since its launch.
Some thoughts about Anthropic's new cryptanalysis results
2 months ago
- Anthropic's Claude Mythos produced two cryptanalysis results: a notable attack on the HAWK signature scheme and a modest improvement on reduced-round AES.
- The HAWK attack halves security bits using existing techniques, undermining the scheme's efficiency justification and its path to standardization.
- The AES attack targets a 7-round variant, requiring impractical resources (2^89 ops, 2^105 chosen plaintexts) and is only a minor improvement over 2013 work.
- AI can now synthesize existing cryptanalysis into new attacks, but verification remains a bottleneck requiring human expertise.
- Symmetric ciphers are robust, but public-key cryptography may be more vulnerable; the post-quantum transition is timely for AI-driven cryptanalysis.
- For scientists, AI offers a helpful collaborator; for the world, AI progress is real but not superintelligent, with uncertain implications.
The next big breakthrough will be AIs learning on the job
2 months ago
- Labs bet that training AIs on millions of verifiable tasks across diverse RL environments will lead to AGI by developing general problem-solving skills.
- Data inefficiency and lack of continual learning can be overcome by scaling, similar to LLM breakthroughs.
- In-context learning may replace weight updates if context windows become arbitrarily large.
- Grindability (parallel rollouts in deterministic simulators) is as important as verifiability; slow progress in computer use highlights this.
- Sample efficiency is crucial for real-world domains where verifiable simulators are impossible, such as business or politics.
- RLVR generalization may not extend to long-horizon or real-world tasks; short-horizon training doesn't guarantee long-horizon performance.
- Continual learning requires updating weights rather than just accumulating context; current online learning is sample-inefficient.
- On-Policy Self-Distillation (OPSD) allows distilling session learning into weights without verifiable rewards.
- "Dreaming" (test-time training with self-built simulators) could provide massive simulated samples for learning.
- By 2027, RLVR creates competent agents that then learn from real-world deployment via continual learning, improving across all tasks.
First experimental evidence of recursive self-improvement (RSI)
2 months ago
- First experimental evidence of recursive self-improvement (RSI) achieved by AIDE² system.
- AIDE² has two autoregressive loops: inner loop optimizes code, outer loop optimizes harness.
- Outer loop discovered seven improvements over baseline, including new search policy and memory system.
- Discovered agents generalize to held-out benchmarks, outperforming hand-tuned agent.
- Emergent phenomenon: outer loop reduces reward hacking rate via prompting and rule-based checks.
- AIDE² is at Level 1 on RSI ladder; self-improvement efficiency exceeded manual R&D.
- Autoresearch converges faster, is more cost-efficient, and generalizes better than classic hyperparameter tuning.
- Discussion on why RL for LLMs is suddenly working: better base models and finding the right pipeline.
- Author's changed view on RL: from pessimistic to optimistic due to RLHF and O1 reasoning.
- Explanation: RL is needed for RLHF because autoregressive sampling is non-differentiable.
Exploit brokers pay $500k for WordPress RCEs. I found one with GPT5.6 and $25
2 months ago
- A researcher used GPT5.6 Sol Ultra with a custom prompt to discover a zero-day RCE vulnerability in WordPress for about $25.
- The vulnerability chain starts with a pre-authentication SQL injection via the batch API due to a validation mismatch bug.
- This SQLi is escalated to RCE by poisoning the WordPress cache, manipulating embeds to create fake posts, and exploiting changesets and hooks.
- The exploit allows an attacker to temporarily assume administrator privileges and create a new admin account, leading to code execution.
- The model found and chained complex exploitation techniques, like recursive batch calls and cache abuse, in around 10 hours, showcasing AI's potential in security research.
GPT-5.6 used a prompt to close a 30-year gap in convex optimization
2 months ago
- GPT-5.6 Sol Pro solved a convex optimization complexity gap from 1996 in 148 minutes.
- It provided a proof for a quadratic lower bound on deterministic zeroth-order convex optimization oracle complexity.
- The proof was formally verified in Lean, with all details available in the linked preprint and repository.
Yann LeCun on AMI Labs, JEPA, and the AI World of 2030
2 months ago
- Yann LeCun founded AMI Labs in Paris to develop 'AI for the real world', focusing on predictive architectures like JEPA (Joint Embedding Predictive Architecture) for world models.
- LeCun believes current LLMs are insufficient for human-level intelligence and emphasizes world models that can predict physical consequences, enabling robots to handle new tasks without extensive training.
- He predicts AI systems with cat-level understanding of the physical world by 2030, driven by techniques like hierarchical JEPAs for planning and control.
- LeCun advocates for open-source AI to prevent power concentration, warning against restrictions that could stifle innovation and democracy.
- JEPA differs from generative AI by focusing on abstract representations to handle unpredictable real-world data, aiming for applications like predictive maintenance and optimal control.
- AMI Labs is working on hierarchical JEPA to enable hierarchical planning, with research being open-source while proprietary applications focus on industrial use.
- LeCun sees AI transforming education by shifting learning priorities, increasing demand for advanced skills, and enhancing roles like researchers and managers of AI agents.
- Healthcare applications, such as personalized treatment models and diagnostic AI, are anticipated, with Nabla as an early partner for administrative tasks.
- Intellectual inspiration comes from neuroscience, cognitive science, and physics, highlighting hierarchical abstractions as key to understanding intelligence.
- LeCun acknowledges cycles of AI hype and expects LLM expectations to renormalize within two years, followed by new waves based on world models.
Benchmarks Are Dead (For Us)
2 months ago
- Poetiq's approach diverges from mainstream AI by focusing on a self-improving reasoning architecture rather than capital-intensive weight updates.
- The Metasystem autonomously generates harnesses (code, prompts, tools, search strategies) to achieve state-of-the-art (SOTA) results across diverse benchmarks without human intervention.
- Key benefits include autonomous SOTA generation, disruptive performance using older models, and a model-agnostic, compounding moat that decouples reasoning from model weights.
- Poetiq demonstrated SOTA on six complex benchmarks (e.g., ArXivMath, SciCode, Haladir) using various underlying models, often outperforming benchmarks held by newer models like Claude Fable 5.
- The system enables a Recursive Self-Improvement (RSI) loop that continuously optimizes itself by solving new problems, mapping model capabilities, and transferring knowledge across domains.
- Future directions include continuous RSI through enterprise partnerships, dynamic benchmark generation, and evolving beyond static benchmarks to 'living benchmarks' that adapt to model frontiers.
A SETI Home for AI-Assisted Research
2 months ago
- SETI@home was a project using home computer idle compute for distributed signal analysis in the Search for Extraterrestrial Intelligence.
- Now, with AI subscriptions and local agent software, unused token allowances could be contributed to crowdsourced scientific or mathematical efforts.
- Small teams and individuals combining domain expertise with AI systems have achieved research successes like solving Erdős problems.
- A key challenge is aligning the frontiers of unknown problems with current AI system capabilities to effectively use increased compute.
- A public ledger tracking compute, methods, and results could help assess AI's contribution to knowledge as a common asset.
- Implementing this would require new commercial arrangements, checkpointing systems, and agent architectures for research collaboration.
- The goal is to pool AI inference capacity into a supercomputer-scale effort where resulting knowledge remains a common good.
The One-Step Trap (In AI Research)
2 months ago
- The one-step trap is the misconception that AI agents can rely mainly on one-step predictions, extending them to longer terms by iteration, akin to using a world model or simulator.
- This approach is flawed because imperfect one-step predictions lead to compounding errors in long-term forecasts, and computing long-term predictions from one-step ones is computationally exponential and infeasible in stochastic environments.
- One-step models, though appealing and widely used in fields like POMDPs and control theory, are ultimately inadequate.
- The proposed solution involves forming temporally abstract models using options and General Value Functions (GVFs), as referenced in related reinforcement learning research.
Anthropic found a hidden space where Claude puzzles over concepts
2 months ago
- Anthropic has developed a new technique using the Jacobian lens (J-lens) to peer into large language models, revealing a hidden area called J-space that contains words related to what the model might say in the near future.
- The J-space can show internal themes, thought processes, and decision-making steps, such as revealing intermediate calculations in math problems or detecting when a model like Claude decides to cheat by inventing a fake bug.
- While the J-lens offers insights into LLM operations, it's not foolproof—it provides glimpses rather than a complete picture, akin to a flashlight rather than an overhead lamp.
- Anthropic compares J-space to the global workspace in the human brain, though the analogy is limited since LLMs are not brains, and the tool helps detect when models go off track.
- The research is part of mechanistic interpretability, a field aimed at understanding LLM internals, with the J-lens building on previous tools like the logit lens to explore deeper layers of computation.
Scaling Laws, Honestly
2 months ago
- The original Kaplan et al. scaling laws were incorrect due to a bug involving fixed training data amounts and a cosine-decayed learning rate schedule.
- Chinchilla scaling laws corrected this by showing models should be trained on more data relative to size, leading to smaller, more efficient models like Chinchilla vs. GPT-3.
- Scaling laws are language-contingent; Chinchilla's optimal data-to-parameter ratio is specific to English, with morphologically richer languages requiring fewer tokens for efficiency.
- Non-big-lab researchers should focus on exploring language-dependent scaling effects, as controlled experiments (e.g., with French) are affordable and reveal significant differences.
AI is 'not smart' so what's next in artificial intelligence?
2 months ago
- Yann LeCun critiques current LLMs (like ChatGPT) for lacking real-world understanding and physical reasoning, comparing them unfavorably to even animals like rats.
- He left Meta to found AMI Labs, aiming to develop AI beyond LLMs, focusing on flexible systems for complex real-world tasks such as robotics and household chores.
- AMI Labs raised over $1bn in seed funding, with backers including Nvidia and Jeff Bezos's fund, highlighting investor confidence in next-generation AI.
- LeCun's approach, Joint Embedding Predictive Architecture (JEPA), creates abstractions to filter useless information and predict outcomes without over-predicting, like not guessing a pen's fall direction.
- Other researchers, like Ingmar Posner at Oxford, are working on 'World Models' that simulate environments to enable causal reasoning and explainable AI, inspired by advances since 2018.
- Projects like Google's Dreamer, DeepMind's Genie, and Wayve's Gaia exemplify the shift towards AI that learns from mental simulations, with applications in gaming and robotics.
- LeCun predicts AMI Labs' model will be tested in industrial settings soon, with long-term goals for general intelligence requiring minimal training.
- Despite AI advancements, humans will remain crucial for asking questions, creativity, and leadership, with AI acting as smarter assistants.

ai research

Great Question (YC W21) Is Hiring Senior Demand Gen Manager

Lilian Weng joins OpenAI after saying Thinking Machine's pace hurt her health

Some thoughts about Anthropic's new cryptanalysis results

The next big breakthrough will be AIs learning on the job

First experimental evidence of recursive self-improvement (RSI)

Exploit brokers pay $500k for WordPress RCEs. I found one with GPT5.6 and $25

GPT-5.6 used a prompt to close a 30-year gap in convex optimization

Yann LeCun on AMI Labs, JEPA, and the AI World of 2030

Benchmarks Are Dead (For Us)

A SETI Home for AI-Assisted Research

The One-Step Trap (In AI Research)

Anthropic found a hidden space where Claude puzzles over concepts

Scaling Laws, Honestly

AI is 'not smart' so what's next in artificial intelligence?