Hasty Briefsbeta

All tags

#llms

165 stories total

Bilingual

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad
a year ago
- Current benchmarks for large language models (LLMs) like MathArena focus on numerical answers but neglect rigorous reasoning and proof generation.
- A new evaluation tested state-of-the-art reasoning models on the 2025 USAMO problems, revealing they struggled significantly with less than 5% average performance.
- Detailed analysis of reasoning traces identified common failure modes and unwanted artifacts from model training strategies.
- The study concludes that current LLMs are inadequate for rigorous mathematical reasoning, highlighting the need for improvements in reasoning and proof generation.
Evaluating Agent-Based Program Repair at Google
a year ago
- Agent-based program repair uses LLMs to automatically fix complex bugs by combining planning, tool use, and code generation.
- The paper evaluates agent-based repair in an enterprise context using 178 bugs from Google's issue tracking system (78 human-reported, 100 machine-reported).
- Passerine, an agent similar to SWE-Agent, achieves a 73% plausible patch rate for machine-reported bugs and 25.6% for human-reported bugs using Gemini 1.5 Pro.
- Manual examination shows 43% of machine-reported and 17.9% of human-reported bugs have semantically equivalent patches to the ground-truth.
- The study highlights differences in bug distribution (language diversity, size, spread of changes) between Google's dataset and the open-source SWE-Bench.
Not all AI-assisted programming is vibe coding
a year ago
- Vibe coding is a new form of coding where developers rely heavily on LLMs, often without reviewing the generated code.
- Andrej Karpathy coined the term 'vibe coding' to describe a hands-off approach where developers let LLMs handle most of the coding tasks.
- Vibe coding is not the same as responsible AI-assisted programming, which involves reviewing and understanding the code before committing it.
- Professional software development requires considering factors like performance, security, and maintainability, which vibe coding often overlooks.
- Vibe coding is best suited for low-stakes projects, prototypes, or personal tools where the consequences of bugs are minimal.
- Security and privacy are major concerns with vibe coding, especially when dealing with sensitive data or APIs.
- Vibe coding can lower the barrier to entry for beginners and help experienced developers build intuition about LLM capabilities.
- Sandboxed environments like Claude Artifacts provide a safer way to experiment with vibe coding by limiting potential harm.
- The article encourages responsible use of LLMs in coding, distinguishing between vibe coding and professional software development.
LLMs understand nullability
a year ago
- Large language models (LLMs) like ChatGPT, Claude, and DeepSeek can write code in many domains, making programming accessible to non-technical users.
- Key questions remain about LLMs' ability to write correct code independently and whether they truly 'understand' the code they generate.
- Understanding in LLMs is measured through internal representations and 'thought processes,' which can be studied via model activations.
- Code properties, such as nullability (whether a variable can be null), are easier to study rigorously than natural language concepts due to static analysis tools.
- Experiments show that LLMs learn to infer nullability rules, with larger models performing better on complex type inference tasks.
- A 'nullability probe' was developed to measure internal model states, revealing how LLMs represent and reason about nullable variables.
- Models' understanding of nullability improves with training, but smaller models may regress in performance as training continues.
- The study provides insights into LLMs' internal representations of programming concepts, paving the way for future research on higher-level code understanding.
LLMs flip the script on technology diffusion
a year ago
- LLMs reverse the traditional top-down technology diffusion pattern by benefiting individuals more than corporations or governments.
- ChatGPT is the fastest-growing consumer application with 400 million weekly users, offering broad quasi-expertise across many domains.
- Individuals gain significant new capabilities (coding, legal, research, analytics, content creation) without needing specialized expertise.
- Organizations face muted benefits due to complexity, coordination, legacy systems, security, compliance, and bureaucratic inertia.
- LLMs' versatility is offset by their shallowness and fallibility, limiting transformative impact on structured organizations.
- Current LLM performance is highly accessible and cheap, but future scaling could reintroduce inequality if money buys superior models.
- The AI revolution's rapid, widespread availability contrasts with sci-fi predictions of top-secret government-controlled AI.
- The future is already here and widely distributed, empowering individuals unprecedentedly.
Vim is more useful in the age of LLMs
a year ago
- Vim remains useful in the age of LLMs despite initial assumptions that LLMs would reduce its utility.
- LLMs assist with code generation, but Vim excels in tasks like navigating, yanking text, and debugging, which are still essential.
- Vim's efficiency in a hybrid regime where understanding and editing code manually is necessary makes it more valuable.
- LLMs make learning Vim easier by providing command history and generating complex commands on demand.
- Examples include generating scripts for copying GitHub links and markdown blocks, enhancing productivity with minimal effort.
- The future of coding may involve more bespoke software tailored to individual needs, increasing the value of customizable tools like Vim.
LLMs Don't Reward Originality, They Flatten It
a year ago
- Originality in content is not rewarded by LLMs unless it becomes part of a consensus.
- LLMs tend to flatten new ideas, absorbing them into general knowledge without proper attribution.
- The concept of 'LLM flattening' describes how LLMs simplify and summarize content, favoring consensus over originality.
- Micro LLM flattening occurs at the topic level, where LLMs reshape knowledge to fit the most authoritative pattern.
- Macro LLM flattening happens over time, reducing exposure to new ideas as LLMs trend towards dominant information.
- LLMs struggle to connect new concepts with their originators, often hallucinating definitions or ignoring unique terms.
- Practical strategies to increase visibility include labeling ideas clearly, adding brand mentions, and distributing content widely.
- Originality is still valuable but requires repetition and network support to gain traction in AI-driven search.
Reasoning Models Can Be Effective Without Thinking
a year ago
- Recent LLMs have improved reasoning capabilities by including explicit thinking processes in generation.
- The paper questions the necessity of explicit thinking, showing that bypassing it (NoThinking) can be effective.
- NoThinking outperforms Thinking across seven reasoning datasets, especially in low-budget settings.
- Performance of NoThinking becomes more competitive with pass@k as k increases.
- A parallel scaling approach using NoThinking to generate N outputs independently and aggregating them is highly effective.
- The method outperforms baselines with similar latency and is comparable to Thinking with longer latency (up to 9x).
- The research encourages reconsidering the necessity of lengthy thinking processes for strong reasoning performance.
LLMs Do Not Predict the Next Word
a year ago
- LLMs are initially trained to predict the next token in a sequence, a process known as the next-token objective.
- Instruction finetuning is used to adapt LLMs for specific tasks by training them on datasets designed for prompting, improving zero-shot learning capabilities.
- Reinforcement Learning from Human Feedback (RLHF) is a key training step where LLMs are optimized to produce outputs that humans prefer, moving beyond simple next-token prediction.
- RLHF involves two main steps: reward modeling, where a model learns to predict human preferences, and proximal policy optimization (PPO), which adjusts the LLM to maximize these rewards while staying close to its original behavior.
- LLMs can be viewed as agents that take actions (producing tokens) to maximize rewards, similar to how chess-playing models choose moves to win games.
- The concept of AI agents extends LLMs by mapping their token outputs to real-world actions, enhancing their utility beyond text generation.
- Despite their capabilities, LLMs trained with RLHF can sometimes produce outputs that seem good to humans but are actually flawed, a phenomenon known as reward hacking.
- The training and capabilities of LLMs suggest they are more than just next-token predictors; they are complex systems optimized for various objectives, including human appeal and task performance.
Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance
a year ago
- Study examines the impact of politeness in prompts on LLM performance across English, Chinese, and Japanese.
- Impolite prompts often lead to poor performance, but excessive politeness does not always improve results.
- Optimal politeness levels vary by language, indicating LLMs reflect human cultural norms and linguistic influences.
- Findings emphasize the importance of considering politeness in cross-cultural NLP and LLM applications.
Teaching LLMs how to solid model
a year ago
- LLMs can create CAD models for simple 3D mechanical parts and are expected to improve rapidly.
- An AI mechanical engineer could handle tasks like material selection, design for manufacturing, and CAD model improvement.
- Training generative models on existing CAD files is a promising approach, with research focusing on diffusion and transformer architectures.
- LLMs can generate solid models using programmatic interfaces like OpenSCAD, leveraging their code generation capabilities.
- OpenSCAD allows users to write scripts that render into solid CAD models, making it suitable for LLM-driven modeling.
- LLMs can utilize general mechanical engineering knowledge, such as correctly sizing holes for bolts.
- An evaluation pipeline was developed to test LLMs' ability to generate CAD models via OpenSCAD, comparing generated STL files to reference models.
- Geometric checks in the evaluation include volume difference, bounding box alignment, chamfer distance, and Hausdorff distance.
- Frontier models like Sonnet 3.5 and Sonnet 3.7 show significant performance improvements in CAD modeling tasks.
- Start-ups like AdamCad and Zoo.dev are developing text-to-CAD products, though initial evaluations show LLMs generating OpenSCAD code outperform Zoo.dev's API.
- Future improvements for LLM-driven CAD modeling include better spatial reasoning, iterative debugging tools, and parameter adjustment UIs.
- GenCAD is expected to become a common tool for mechanical engineers within 6-24 months, with long-term potential to revolutionize CAD suites.
The Future of MCPs
a year ago
- MCPs (Model Context Protocols) are standardized APIs that connect external data sources or applications to large language models (LLMs) like ChatGPT or Claude.
- MCPs enable LLMs to access real-time data, take actions on the web, and function more like agents than static chatbots.
- Two main types of MCP Servers exist: developer-focused tools (e.g., Cursor, Claude Code) and web/action-oriented tools for real-world tasks (e.g., booking events, sending emails).
- The author built two experimental MCP servers: GPT Learner (a developer tool) and a prediction market MCP connected to betsee.xyz.
- Current limitations of MCPs include rough user experience, lack of broad client support, security risks, and manual installation processes.
- MCP clients (e.g., ChatGPT, Claude) hold significant power, controlling which tools users see and how responses are surfaced.
- MCPs are expected to evolve into a framework resembling a combination of search engines and mobile app stores, with major LLM providers acting as gatekeepers.
- Future opportunities include MCP wrapper/server packs, affiliate shopping engines, MCP-first content apps, and enterprise private MCP solutions.
- Specialized MCP clients tailored to specific industries (e.g., travel, HR) are likely to emerge.
- The author predicts that most users will rely on default AI experiences with pre-installed MCPs, while power users may opt for customizable open-source clients.
Docker Model Runner Brings Local LLMs to Your Desktop
a year ago
- Docker Model Runner is a new beta feature in Docker Desktop 4.40 for Apple silicon-powered Macs, enabling developers to run LLMs and other AI models locally.
- Running LLMs locally with Docker Model Runner ensures data privacy, reduces latency, and lowers costs by eliminating the need for external cloud APIs.
- Docker Model Runner packages LLMs as Open Container Initiative (OCI) Artifacts, allowing integration into CI/CD pipelines using familiar Docker CLI commands.
- The feature uses llama.cpp, an open-source C++ library, to efficiently deploy and infer LLMs without GPUs, making it possible to run models on older hardware.
- Docker is partnering with AI leaders like Google, HuggingFace, and Qualcomm to expand the selection of high-quality, optimized models available for local use.
- Future plans include expanding support to more platforms, such as Windows with GPU acceleration, and enabling developers to publish custom models.
- Docker is integrating the Model Context Protocol (MCP), an open standard for connecting AI agents to data sources, tools, and application environments.
- The Docker MCP Catalog provides a centralized way to discover, run, and manage over 100 MCP servers from leading providers.
- The Docker MCP Toolkit offers enterprise-grade management and security features for MCP workflows, including registry access management and secrets handling.
- Docker aims to make AI integration seamless, allowing developers to build, test, and deploy AI applications with the same ease as traditional containerized workflows.
Naur's "Programming as Theory Building" and LLMs replacing human programmers
a year ago
- Peter Naur's 'Programming as Theory Building' argues that programming involves building a 'theory' of the program, not just producing code.
- LLMs lack the ability to build theories as defined by Gilbert Ryle, as they only ingest and reproduce the output of work, not the underlying reasoning.
- Naur emphasizes that understanding and modifying large programs requires continuous, intimate knowledge from the programmers who built them.
- Programming is not merely text production; it involves maintaining a deep understanding of the program's structure and purpose.
- LLMs cannot replace human programmers because they cannot develop the necessary theories or understand the context behind code.
Bad but common LLM criticisms
a year ago
- AI is both overhyped and underhyped, with widespread discussion but limited meaningful impact observed so far.
- LLMs have high variability in output, and user dissatisfaction often stems from mismatched expectations rather than poor performance.
- Effective use of LLMs requires detailed prompts, similar to composing an email, to achieve desired results.
- The definition of AI constantly evolves, with past benchmarks like chess and Go now considered trivial.
- LLMs today can perform tasks that were once multi-year PhD projects, making criticisms based on past standards irrelevant.
- LLMs, like humans, make mistakes, but this doesn't negate their value.
- The 'fancy autocomplete' criticism is outdated, as LLMs' capabilities far surpass simple autocomplete features.
- The rate of improvement for LLMs has slowed, and it's unclear if exponential growth will continue.
- LLMs have ingested nearly all human knowledge, raising questions about future learning from synthetic data.
When ChatGPT Broke an Entire Field: An Oral History
a year ago
- Natural language processing (NLP) has undergone significant changes, especially with the emergence of large language models (LLMs).
- The introduction of the transformer model in 2017 marked a turning point, though its impact wasn't immediately recognized.
- BERT and GPT models shattered performance records, leading to a 'benchmark boom' and debates about their capabilities.
- The release of GPT-3 in 2020 was a pivotal moment, showcasing the potential of LLMs and causing existential crises among researchers.
- ChatGPT's launch in 2022 disrupted the field, making many traditional NLP research areas obsolete and shifting focus to LLM applications.
- The field has seen divisions over the ethical and practical implications of LLMs, with debates around understanding, scale, and corporate influence.
- Researchers are now grappling with the societal impact of LLMs, their limitations, and the future direction of NLP and AI.
Llasa: Llama-Based Speech Synthesis
a year ago
- Explores scaling of train-time and inference-time compute for speech synthesis.
- Proposes LLaSA, a simple framework using a single-layer VQ codec and Transformer architecture aligned with LLMs like LLaMA.
- Shows that scaling train-time compute improves speech naturalness and prosody patterns.
- Demonstrates that scaling inference-time compute enhances emotional expressiveness, timbre consistency, and content accuracy via verifiers.
- Releases checkpoints and training code for TTS models (1B, 3B, 8B) and codec model publicly.
- Compares inference-time scaling results using different evaluation metrics and benchmarks like Ravdess.
- Evaluates models across various sizes and training data amounts, highlighting text comprehension and synthesis quality.
LLMs for Engineering: Teaching Models to Design High Powered Rockets
a year ago
- LLMs are explored for high-powered rocketry design using RocketBench, a benchmark connecting LLMs to rocket simulations.
- Two design tasks tested: target altitude optimization and precision landing challenges.
- State-of-the-art LLMs show strong baseline engineering knowledge but struggle to iterate on designs with simulation results, plateauing below human performance.
- A 7B parameter model enhanced with reinforcement learning (RL) outperforms both SoTA foundation models and human experts.
- RL-trained LLMs can serve as effective tools for complex engineering optimization, extending beyond software development.
Reasons to Write More in an Age When Writing Means Less
a year ago
- LLMs have significantly impacted the internet and professional writing, with many using them silently to speed up work.
- Writing helps in understanding and articulating thoughts clearly; relying on LLMs to write may lead to a loss of human thinking and relevance.
- Writing and publishing online allows individuals to influence the development of powerful technologies like LLMs.
- Tyler Cowen advocates for writing to achieve intellectual immortality, as LLMs can preserve thoughts for future generations.
- Writing about niche or rare knowledge can enhance collective intelligence and the knowledge base of LLMs.
- Documenting forgotten or rare information benefits future historians and AI learning processes.
- Writing is a powerful tool for shaping thoughts and contributing to the collective knowledge, even if individual contributions seem small.
Cell Mates: Extracting Useful Information from Tables for LLMs
a year ago
- LLMs currently lack the ability to effectively encode knowledge from tabular data like survey data beyond published statistical summaries.
- The main challenge is finding a useful representation for tabular data; representing each row as a sentence misses much of the table's knowledge.
- Mechanical distillation techniques are proposed, including creating univariate, bivariate, and multivariate summaries based on table structure.
- The approach involves understanding the data's collection and structure, learning what questions can be asked, and creating mechanical summaries and plots.
- This pipeline could be used for RAGs and supplementing 'world data', with scientific data repositories like Harvard Dataverse suggested as starting points.

firstprev0next

About|Login

#llms

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

Evaluating Agent-Based Program Repair at Google

Not all AI-assisted programming is vibe coding

LLMs understand nullability

LLMs flip the script on technology diffusion

Vim is more useful in the age of LLMs

LLMs Don't Reward Originality, They Flatten It

Reasoning Models Can Be Effective Without Thinking

LLMs Do Not Predict the Next Word

Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance

Teaching LLMs how to solid model

The Future of MCPs

Docker Model Runner Brings Local LLMs to Your Desktop

Naur's "Programming as Theory Building" and LLMs replacing human programmers

Bad but common LLM criticisms

When ChatGPT Broke an Entire Field: An Oral History

Llasa: Llama-Based Speech Synthesis

LLMs for Engineering: Teaching Models to Design High Powered Rockets

Reasons to Write More in an Age When Writing Means Less

Cell Mates: Extracting Useful Information from Tables for LLMs