llms

Bilingual

LLMs and Xfwl4
2 months ago
- LLMs are a tool with positive and negative aspects, including concerns over training data and copyright infringement.
- Environmental and social impacts of LLMs include high resource consumption and secret deals that burden local ratepayers.
- GPU and RAM supply issues are exacerbated by AI demand, making personal hardware upgrades expensive.
- The author uses Claude extensively for research and planning in developing xfwl4, such as analyzing xfwm4 source code.
- Implementation is mostly done manually, with Claude used for validation, code review, and occasional boilerplate code generation.
- Claude is highly effective for bug fixing, often finding issues faster than the author.
- Unit tests in xfwl4 were written by Claude due to the author's distaste for testing.
- The author avoids vibe-coding and focuses on understanding the code for future manual maintenance.
- LLM usage has saved months of time, particularly in problem-solving and debugging, despite ethical discomfort.
- The author provides transparency about LLM use in the xfwl4 development process.
The Project of Software Is Complete
2 months ago
- The project of software is complete; the era of new software categories has ended, and current efforts are refinements rather than innovations.
- Despite the democratization of coding via LLMs ('vibe coding'), most new software is redundant or trivial, not addressing genuine unmet needs.
- The adjacent possible of software is saturated; new apps like habit trackers are terminal leaves, not branches that open new frontiers.
- LLMs are powerful at manipulating symbols but cannot solve physical-world problems (e.g., energy, disease, transportation) that require atoms, not bits.
- Future technological progress must come from material science and engineering, not from more layers of software abstraction.
LLMs can now identify public figures in images
2 months ago
- Multimodal LLMs can extract structured semantic data from images for better categorization, tagging, and searching.
- ChatGPT and Claude refuse to identify public figures like Barack Obama due to AI safety policies and RLHF training, while Gemini has no such hesitation.
- In tests across six LLMs, Gemini consistently outperforms others in accurately identifying public figures, including in challenging scenarios like group photos and costume images.
- Refusals by GPT and Claude to identify public figures can be bypassed using prompt engineering techniques, such as instructing the model to prefix its output.
- The accuracy of LLMs in identifying people varies, with some models hallucinating incorrect identities (e.g., Mistral misidentifying Priscilla Chan as Sheryl Sandberg).
- Gemini's superior performance is attributed to Google's access to vast training data from its search engine, achieving over 90% accuracy in identifying public figures across diverse domains.
- There is an ethical concern about LLMs potentially identifying nonpublic figures in the future as models improve and RLHF rules become laxer.
LLMs are a 400-year-long confidence trick
2 months ago
- LLMs are compared to a 400-year-long confidence trick, rooted in the historical development of mechanical calculators and trust in machine accuracy.
- The con stages involve building trust (through centuries of reliance on machines), exploiting emotions (fear of job loss, flattery from RLHF), and creating a false urgency to adopt LLMs.
- Despite promises of intelligence, LLMs are not truly intelligent, and 95% of AI projects fail to provide ROI, highlighting the gap between hype and reality.
LLMs are bad at vibing specifications
2 months ago
- LLMs are generating formal specifications but often produce trivial, tautological properties that do not verify anything meaningful.
- Many AI-generated specs fail to compile or model-check due to incorrect usage of language features like Boolean modules.
- The true value of formal methods lies in subtle properties, but LLMs tend to generate only obvious, weak properties.
- Using LLMs for formal methods may still require expertise, raising questions about how much they lower the skill threshold.
- A book giveaway for 'Logic for Programmers' is announced with multiple timezone-specific coupons to address previous issues.
Entropic Thoughts
2 months ago
- Top-of-the-line LLMs on easy tasks generate code about 10% more complicated than necessary.
- This complexity is often accepted because it solves the immediate problem and is locally contained.
- An example: a 24-line function to convert a string to a safe HTTP header value had a dead branch due to implicit assumptions, which the author rewrote in 15 lines.
- The author questions the long-term impact of consistently accepting unnecessarily complex code.
- A possible counter-argument is that LLMs will improve, but the author remains unconvinced.
<antirez>
2 months ago
- Critics who previously dismissed LLMs as fundamentally flawed now claim that reasoning models like OpenAI o1 and DeepSeek R1 are not 'just LLMs,' but this is false.
- DeepSeek R1 is a pure decoder-only autoregressive model using the same next-token prediction criticized earlier, with no explicit symbolic reasoning.
- R1 Zero achieved strong reasoning capabilities through reinforcement learning and chain-of-thought generation without any supervised fine-tuning.
- The S1 paper shows that as few as 1,000 examples can enable complex reasoning, indicating that pre-training already encodes the necessary representations.
- Reasoning models are fundamentally LLMs, and those who declared LLMs a dead end were wrong; attempts to rewrite history are unacceptable.
<antirez>
2 months ago
- By 2025, most AI researchers stopped claiming LLMs are stochastic parrots due to accumulating functional evidence.
- Chain-of-thought (CoT) improves LLM output by enabling internal sampling and reinforcement learning for token sequences.
- Reinforcement learning with verifiable rewards overcomes scaling limits on token count, with potential for continued improvement.
- Programmer resistance to AI-assisted coding has decreased as LLMs provide useful code and hints with acceptable return on investment.
- Some researchers explore alternatives to Transformers, but LLMs may still reach AGI without new paradigms.
- Critics who now credit CoT for fundamentally changing LLMs are misleading; the architecture and token prediction remain the same.
- The ARC-AGI test has become solvable by LLMs with extensive CoT, transitioning from an anti-LLM test to validation.
- The fundamental AI challenge for the next 20 years is avoiding extinction.
LLMs aren't remotely like compilers or power tools
2 months ago
- LLMs are fundamentally different from predictable tools like compilers or power tools, which are designed for reliability and consistency.
- Delegation is a more accurate metaphor for LLMs because they involve stochastic, unpredictable outputs similar to human agents, leading to a principal-agent problem.
- Natural language interfaces of LLMs are inherently ambiguous and imprecise, making them less reliable than structured interfaces like programming languages or physical controls.
- LLMs mimic human-like tasks but are profoundly strange and not directly comparable to existing entities, as they operate based on complex neural network processes.
- While LLMs can be useful in specific, narrow situations, their current societal discourse is often misleading and thought-terminating.
DSLs Enable Reliable Use of LLMs
2 months ago
- Domain-Specific Languages (DSLs) enhance LLM reliability by providing clear boundaries and constraints, reducing variability in code generation.
- DSLs act as a source of truth for software systems, enabling LLMs to serve as natural language interfaces and iterative partners in design.
- Abstractions and semantic models, like Tickloom for distributed systems, ground LLM prompts in existing vocabulary, improving accuracy and reducing hallucinations.
- LLMs operate in two phases: as brainstorming partners during DSL design and as reliable generators once the DSL is established.
- DSLs support deterministic validation (e.g., parsers or compilers), allowing autonomous agents to self-correct errors without human intervention.
- Examples include using LLMs with PlantUML for presentations and internal DSLs for testing distributed system scenarios, showcasing their versatility.
- Well-factored DSLs produce maintainable, readable artifacts that capture intent, reducing reliance on prompts as the primary source of truth.
Canonization and the Overhang
2 months ago
- David Bessis argues that mathematics overvalues theorem proving while undervaluing creativity, clarity, and teaching, which are critical for progress.
- LLM-powered theorem provers can prove theorems without the beneficial side effects of human collaboration and intuition.
- In software, there's a parallel between code that passes tests and code that is socially constructed for team usability and iteration.
- Canonization is the process of turning one-off solutions into reusable, coherent, and compatible systems or libraries, which is time-consuming but essential for long-term value.
- Software benefits from canonized code through stable abstractions and reliable systems, contrasting with disposable, single-use software that may seem 'bad' by traditional standards.
- The Overhang refers to unrealized potential from past creativity, where connecting existing ideas can yield new solutions, a concept applicable to both mathematics and software.
- LLMs are well-suited for searching the Overhang by memorizing and pattern-matching across large corpora, unlike humans who rely on intuitive leaps.
- Open source software represents an Overhang of canonized work that can be built upon for future development.
- Valuing only theorem proving or coding without canonization and knowledge transmission risks depleting the Overhang and undervaluing critical 'glue work' like cleanup and teaching.
LLM's as a Different Kind of Intelligence
3 months ago
- LLMs represent a different kind of intelligence, distinct from human intelligence.
- Human analogies like 'junior engineer' or 'abstraction layer' are flawed because LLMs lack intent and deterministic reasoning.
- LLMs combine vast knowledge with an inability to understand intent, leading to subtle drift from user goals over time.
- Probabilistic behavior from LLMs challenges the deterministic foundations of software engineering.
- Effective use requires learning LLMs' unique failure modes rather than forcing familiar comparisons.

LLMs and Xfwl4

The Project of Software Is Complete

LLMs can now identify public figures in images

LLMs are a 400-year-long confidence trick

LLMs are bad at vibing specifications

Entropic Thoughts

<antirez>

<antirez>

LLMs aren't remotely like compilers or power tools

DSLs Enable Reliable Use of LLMs

Canonization and the Overhang

LLM's as a Different Kind of Intelligence