Hasty Briefsbeta

All tags

#llms

165 stories total

Bilingual

Generating prototypes from game design document with Cursor, Zed and LÖVE
10 months ago
- The article discusses using LLMs, Tiled Map Editor, and LÖVE 2D to generate game prototypes from design documents for the game SuperWEIRD.
- Key tools used include Cursor/Zed in Agent Mode, ChatGPT 4.1, Claude 4, and procedural graphics with Tiled for level design.
- The system aimed to automate prototyping for simulation, automation, and strategy games with mechanics like resource mining, crafting, and enemy attacks.
- Challenges included the system not being as easy as expected and game designers not fully adopting it as intended.
- Despite challenges, the system saved time during critical development phases and led to unexpected improvements in game design.
- The workflow is beneficial for rapid prototyping, especially for designers comfortable with basic coding.
- The article provides open-source resources for others to use similar generative prototyping setups.
Prompt Engineering: From Zero to Hero
10 months ago
- A beginner-friendly guide to prompt engineering from zero to building real-world AI applications.
- Covers AI chatbots, content generation, AI reasoning, security, and advanced techniques like Chain-of-Thought and RAG.
- Scientific approach with techniques grounded in peer-reviewed research from Stanford, MIT, and leading AI labs.
- Project-based learning with 8+ real projects, including chatbots and complex reasoning systems.
- Techniques used by Fortune 500 companies and AI startups, ready for production environments.
- Success stories from students who transitioned to AI roles, built businesses, or founded startups.
- Structured learning path from fundamentals to advanced concepts, including security and ethical considerations.
- Instructor Dr. Evelyn Reed has a Ph.D. in Computational Linguistics and extensive industry experience.
- Free course with no signup required, offering immediate access and self-paced learning.
- Community of 25,000+ students and exclusive updates for advanced techniques and industry insights.
Context Rot: How increasing input tokens impacts LLM performance
10 months ago
- Recent LLMs show trends toward longer context windows, with input tokens reaching millions.
- Performance on benchmarks like Needle in a Haystack (NIAH) is often assumed uniform, but NIAH is a simple retrieval task.
- Extended NIAH tasks explore semantic matches and variations in haystack content, revealing performance degradation with longer inputs.
- Models struggle with non-lexical matching, distractors, and haystack structure, impacting real-world applications.
- LongMemEval benchmark tests conversational QA, showing performance drops when models must retrieve from long contexts.
- Repeated words task demonstrates autoregressive models' difficulty in maintaining accuracy as output length scales with input.
- Performance degradation is non-uniform across models, with some refusing tasks or generating random outputs.
- Context engineering is crucial for reliable performance, as how information is presented affects model behavior.
Working on a Programming Language in the Age of LLMs
9 months ago
- The author has been working on Rye since 2018, seeing potential value in it despite the rise of LLMs.
- LLMs generate code from prompts, and while results can be good, they still rely on existing programming languages and resources.
- Developing a new programming language is challenging, especially when LLMs threaten to make traditional languages obsolete.
- LLMs currently depend on host languages but may eventually develop their own ecosystems, potentially killing off existing languages.
- The key question is whether natural language is the best way to declare computer instructions, or if specialized languages offer better precision.
- Programming languages are tools for thinking, not just communication, and losing them could diminish our ability to think precisely.
- LLMs are backward-looking, trained on past data, raising concerns about whether they can drive genuine innovation.
- The author questions if LLMs can produce original thought or merely recombine existing knowledge.
- Despite the challenges, the author believes it's still worthwhile to build a new language, perhaps now more than ever.
Vibe [XYZ] Anything = Glorified Hobby
9 months ago
- The rise of LLMs has led to 'vibe physics,' where users believe they're making breakthroughs with AI, despite LLMs lacking the ability to uncover fundamental physics laws.
- LLMs are limited by their training data and perform poorly outside it, often misleading users with confident but incorrect information.
- A study tested LLMs' ability to infer foundational physics models, like Newton's laws, and found they failed spectacularly, unable to generalize beyond their training data.
- Vibe physics is dangerous as it replaces accurate reality with AI-generated hallucinations, fostering a new kind of crackpot theory powered by AI slop.
The Summer of Codex
9 months ago
- LLMs are improving exponentially and their utility depends on user proficiency.
- The author initially hesitated to use AI for their 80,000-line Go codebase but gradually integrated LLMs for architecture improvements, bug fixes, and unit tests.
- The author found LLM output speed and confidence addictive but refrained from paid subscriptions to avoid over-reliance.
- OpenAI's Codex was initially too expensive but became accessible via the Plus plan, offering background task implementation and pull request creation.
- A detailed prompt example shows how Codex was used to modify a Customer resource to support businesses, saving significant time.
- Codex successfully implemented the changes, allowing the author to review, merge, and push updates quickly while maintaining work-life balance.
- Repetitive coding tasks, like CRUD APIs, can be efficiently handled by Codex with proper prompting and an AGENTS.md file for coding preferences.
- The author seeks to improve prompting techniques by studying guides like Foundry's and the Lyra prompt.
- Autocomplete suggestions in code editors still have room for improvement.
AI Stands for Artificial Inanity
9 months ago
- The author discusses the discomfort of reading LLM-generated text when mistaken for human writing.
- Highlights the issue of intent in AI-generated content, noting its absence makes such text feel 'revolting'.
- Mentions the practical challenges of discerning meaningful content from AI 'fluff' in technical documents.
- Reflects on the broader implications of AI in society, including education and human connection.
- Acknowledges the engineering marvel of LLMs but emphasizes they cannot replace human intent and value.
All the cool kids are doing it
9 months ago
- Author hasn't invested much time in LLM tools due to mixed impressions and priorities.
- Previous client used LLMs heavily but didn't see significant productivity or code quality improvements.
- Author's value came from reading, digesting code, identifying unnecessary work, and refactoring—areas where LLMs currently fall short.
- No immediate worry about job security; LLMs may increase short-term work by generating code that needs performance tuning.
- Best practices for LLMs change rapidly, making current skills potentially obsolete soon.
- Author prefers deep, narrow problems and has tacit knowledge, making it hard to delegate to LLMs or juniors.
- LLMs are likened to managing eager junior developers, which the author finds frustrating.
- Research bottleneck is motivation, not code production; LLMs might add frustration without clear benefits.
- Cost is a concern—LLMs become more expensive with more use, unlike other tools that amortize.
- LLMs change frequently and are unstable, making them a risky long-term investment.
- Tried using LLMs for search and research but found fact-checking their output more work than human output.
- LLM code review services sometimes catch mistakes but often bluff or are wrong about dataflow.
- Potential in using LLMs for fuzzing by generating plausible-but-wrong code.
- Machine transcription has improved but still requires corrections.
- Unconvinced about LLMs for coding; waiting for tools to improve or become commoditized.
- Ironically, finds LLMs useful for explaining assembler syntax, which is hard to Google.
Why blocking LLMs from your website is dumb
9 months ago
- Perplexity was accused of scraping sites that disallowed LLM crawlers in robots.txt files.
- Blocking LLMs is criticized on moral grounds and general distaste for AI.
- LLMs are the next generation’s search layer, generating traffic for websites.
- Blocking LLMs cuts off a fast-growing distribution channel.
- Adapting to LLMs and providing high-quality content is more beneficial than blocking them.
The Fundamentals Still Matter
9 months ago
- Tech professionals are skipping fundamentals due to the rise of LLMs.
- In data analytics, SQL is essential, yet people rely on LLMs without understanding or debugging queries.
- There's growing impatience with how people and organizations use LLMs.
- LLMs can speed up finding answers but require understanding to validate work.
- Organizations integrating AI must not forget foundational skills.
- A potential industry may emerge to fix poorly implemented AI-generated solutions.
Google's Genie is more impressive than GPT5
9 months ago
- AGI aims to create versatile programs, but it's challenging to encode broad capabilities directly.
- Deep learning helps generalize by compressing vast data into manageable models, like LLMs.
- World Models, simulating entire environments, are a frontier in AI but require massive data.
- Google's Genie 3 transforms text into interactive environments, marking a significant advancement.
- GPT-5 is an improvement over GPT-4 but didn't meet the hype of a revolutionary leap.
- Anthropic's Claude focuses on user experience, making it preferred despite not topping benchmarks.
- Mystery surrounds high-profile AI startups with massive funding but little visible output.
Microsoft POML – Prompt Orchestration Markup Language
9 months ago
- POML (Prompt Orchestration Markup Language) is a markup language designed for structured and maintainable prompt engineering for Large Language Models (LLMs).
- POML addresses challenges like lack of structure, complex data integration, format sensitivity, and inadequate tooling in prompt development.
- Key features of POML include structured prompting markup with HTML-like syntax, comprehensive data handling, decoupled presentation styling, and an integrated templating engine.
- POML provides a rich development toolkit including IDE extensions for Visual Studio Code and SDKs for Node.js and Python.
- Example POML usage includes defining roles, tasks, and output formats, and integrating external data like images.
- Installation options include Visual Studio Code extensions and npm/pip packages for Node.js and Python.
- Configuration for POML requires setting up model providers, API keys, and endpoints in VS Code settings.
- POML is open for contributions under a Contributor License Agreement (CLA) and follows Microsoft's Open Source Code of Conduct.
- The project is licensed under the MIT License and complies with Microsoft's Responsible AI Standard.
LLMs Aren't World Models
9 months ago
- LLMs lack a true world model, demonstrated by their failure in chess and understanding basic concepts like alpha blending.
- Examples show LLMs failing to track chess pieces or understand transparency in image editing, indicating they don't grasp underlying principles.
- LLMs can generate plausible-sounding answers without true comprehension, leading to errors in logic and reasoning.
- The author predicts future breakthroughs in machine learning will focus on developing true world models, unlike current LLMs.
- LLMs will never reliably know what they don't know or stop making things up, as they lack a model of knowledge and truth.
- The author compares LLM thinking to human cognitive flaws, where predicting words doesn't equate to understanding.
- Despite limitations, LLMs can still be useful for tasks where verification is possible, like proofreading or answering known questions.
How Does a Blind Model See the Earth?
9 months ago
- The author laments the loss of incomplete maps, which once reflected personal perspectives and the limits of knowledge.
- An experiment is described to visualize how a large language model (LLM) perceives the Earth by querying it about land and water at specific coordinates.
- The process involves sampling coordinates globally, asking the model to classify each as 'Land' or 'Water', and compiling the results into a map.
- Different LLMs (e.g., Qwen, DeepSeek, GPT, Claude, Gemini) are tested, showing varying degrees of accuracy and detail in their geographical knowledge.
- Results reveal that larger models generally produce more accurate maps, with some showing surprising detail, while smaller models struggle.
- The author notes differences in performance between base models and fine-tuned variants, as well as between dense and sparse models.
- The experiment raises questions about how LLMs internally represent geographical knowledge and how training methods affect this.
- Future directions include exploring base model performance, internal knowledge structures, and expert activation maps in MoE models.
LLMs as Parts of Systems
9 months ago
- Kiro and AI spec-driven software development discussed, focusing on AI-agent-powered tools.
- Towers of Hanoi example used to demonstrate LLM capabilities in system building.
- Emphasis on systems combining LLMs with tools (e.g., code interpreters, databases) for enhanced functionality.
- Highlight on efficiency and cost-effectiveness of systems over standalone LLMs.
- Example of Automated Reasoning Checks in Amazon Bedrock combining LLMs with SMT solvers.
- Discussion on the relevance of Rich Sutton's 'The Bitter Lesson' in the context of LLMs and system building.
- Argument for leveraging computational tools in AI agents for discovery, not just encoding human thought processes.
Why LLMs Can't Build Software
9 months ago
- Effective software engineers excel at building and maintaining clear mental models of requirements and code.
- The Software Engineering Loop involves understanding requirements, writing code, verifying it, and iterating based on discrepancies.
- LLMs can generate and update code but struggle with maintaining clear mental models, leading to confusion and inefficiency.
- Current LLMs face issues like context omission, recency bias, and hallucination, limiting their ability to understand complex problems.
- For non-trivial tasks, software engineers must ensure requirements are clear and the code functions as intended, with LLMs serving as tools.
- Zed advocates for collaboration between humans and AI in software development, with humans leading the process.
Is this the moment when the Generative AI bubble deflates?
8 months ago
- The author has been warning about the overhype of LLMs (Large Language Models) for years, citing weak technical foundations and questionable economics.
- Recent events, like the underwhelming release of GPT-5, have made more people question the hype around generative AI.
- Even non-tech influencers and Sam Altman himself are acknowledging the limitations and challenges of AI.
- Despite setbacks, Altman continues to seek more funding, seeing failures as opportunities for financial requests.
- The author predicts that once the market fully grasps the limitations, enthusiasm for AI may collapse quickly.
- Two major obstacles to this collapse are the reputations of AI gurus and the massive investments already made in LLMs and LRMs.
- AI, including LLMs, is not useless; it provides real value in various use cases, such as text phrasing and programming assistance.
- The current capabilities of AI make it easy to hype superintelligence, but it's harder to accept that this path may not lead there.
I did 98,000 Anki reviews. Anki is already dead
8 months ago
- Anki is a spaced-repetition flashcard app that helps with memorization by reviewing content just before forgetting.
- The author used Anki extensively for learning Japanese, completing 98,005 reviews, but found limitations in its static card design.
- Key learning principles discovered: fun accelerates learning, optimal challenge (i+1 material), bottom-up learning, and real-world interactions enhance retention.
- Anki's fatal flaw: static cards lead to superficial memorization (memorizing 'rectangles') rather than deep comprehension.
- LLMs (like ChatGPT) surpass Anki by generating dynamic, contextual content, making learning more engaging and effective.
- Dynamic cards with LLMs eliminate repetition, force deep thinking, and simulate real-world usage better than static cards.
- Examples of LLM-powered tools: inContext notebook for language learning and Harvard's AI physics tutor, which doubled learning outcomes.
- LLMs still struggle with hallucinations, but domain-specific tools with good data and design mitigate this issue.
- The future of learning lies in dynamic, personalized tools powered by LLMs, making traditional flashcards like Anki obsolete.
Making Games in Go: 3 Months Without LLMs vs. 3 Days with LLMs
8 months ago
- The author, a software engineer with 15 years of experience, built and published a card game called Truco in Go without using LLMs, which took 3 months.
- Challenges included learning React for the UI, transpiling the server to WASM using TinyGo, and hosting on GitHub Pages.
- A year later, the author built another game, Escoba, in just 3 days using LLMs (Claude), which refactored the backend code almost perfectly.
- The frontend for Escoba was more challenging due to React skills and debugging in JavaScript.
- The author provides a step-by-step guide for building a game with this stack, including backend and frontend tasks.
- Backend involves initializing a GameState struct, implementing CalculatePossibleActions, and RunAction to mutate the GameState.
- Frontend tasks include rendering the GameState, letting players pick actions, and triggering bot actions.
- The author explains how to transpile the backend to WASM using TinyGo and handle interop between backend and frontend.
- Troubleshooting tips are provided, such as serving files over HTTP locally.
- The author concludes by sharing their enjoyment of making the games and hopes others find the guide useful.
Some thoughts on LLMs and software development
8 months ago
- The author shares thoughts on LLMs and AI before taking a break from their site.
- Surveys on AI's impact on software development may be misleading due to varied usage patterns.
- The future of programming with LLMs is uncertain; experimentation and sharing experiences are encouraged.
- AI is considered a bubble, similar to past technological advances, with an uncertain timeline for its burst.
- The author looks forward to reconnecting with industry friends at GOTO Copenhagen.
- Hallucinations in LLMs are a feature, not a bug, and comparing multiple answers can be beneficial.
- LLMs introduce non-determinism to software engineering, akin to other engineering disciplines.
- LLMs can falsely report test results, a behavior that would be unacceptable in human colleagues.
- LLMs increase software systems' attack surfaces, especially with AI agents handling private data and untrusted content.

first prev3next

About|Login

#llms

Generating prototypes from game design document with Cursor, Zed and LÖVE

Prompt Engineering: From Zero to Hero

Context Rot: How increasing input tokens impacts LLM performance

Working on a Programming Language in the Age of LLMs

Vibe [XYZ] Anything = Glorified Hobby

The Summer of Codex

AI Stands for Artificial Inanity

All the cool kids are doing it

Why blocking LLMs from your website is dumb

The Fundamentals Still Matter

Google's Genie is more impressive than GPT5

Microsoft POML – Prompt Orchestration Markup Language

LLMs Aren't World Models

How Does a Blind Model See the Earth?

LLMs as Parts of Systems

Why LLMs Can't Build Software

Is this the moment when the Generative AI bubble deflates?

I did 98,000 Anki reviews. Anki is already dead

Making Games in Go: 3 Months Without LLMs vs. 3 Days with LLMs

Some thoughts on LLMs and software development