Hasty Briefsbeta

All tags

#llm

310 stories total

Bilingual

Transformer Lab
a year ago
- Transformer Lab is supported by Mozilla through the Mozilla Builders Program.
- It is an open-source platform for building, tuning, and running Large Language Models (LLMs) locally without coding.
- Aims to enable software developers to integrate LLMs into their products without Python or ML expertise.
- Features include one-click downloads of popular models like Llama3, Phi3, Mistral, and more from Huggingface.
- Supports finetuning across different hardware, including Apple Silicon (MLX) and GPUs (Huggingface).
- Offers RLHF and preference optimization techniques like DPO, ORPO, SIMPO, and reward modeling.
- Compatible with Windows, MacOS, and Linux operating systems.
- Provides chat functionalities, preset prompts, chat history, and batch inference.
- Supports multiple inference engines: MLX, Huggingface Transformers, vLLM, and Llama CPP.
- Includes evaluation tools like Eleuther Harness, LLM as a Judge, and red teaming evals.
- Features RAG (Retrieval Augmented Generation) with drag-and-drop file support.
- Allows dataset building from HuggingFace or custom datasets via drag-and-drop.
- Offers embedding calculations, a full REST API, and cloud or local deployment options.
- Supports model conversion across platforms (Huggingface, MLX, GGUF) and plugin extensions.
- Includes prompt editing, system message adjustments, and inference logs for transparency.
Teuken-7B-Base and Teuken-7B-Instruct: Towards European LLMs
a year ago
- Introduction of Teuken-7B-Base and Teuken-7B-Instruct, multilingual LLMs supporting all 24 official EU languages.
- Models trained on ~60% non-English data with a custom multilingual tokenizer to address English-centric LLM limitations.
- Detailed development principles include data composition, tokenizer optimization, and training methodologies.
- Competitive performance demonstrated on European versions of ARC, HellaSwag, MMLU, and TruthfulQA benchmarks.
FastMCP: The fast, Pythonic way to build MCP servers and clients
a year ago
- FastMCP is a Pythonic interface for building and interacting with MCP (Model Context Protocol) servers.
- MCP servers expose data and functionality to LLM applications in a secure, standardized way.
- FastMCP simplifies MCP server creation with decorators, handling protocol details and server management.
- Key features include tools (for actions), resources (for data), and prompts (for interaction patterns).
- FastMCP 2.0 introduces advanced features like proxying, composing servers, and generating from OpenAPI specs.
- The Context object provides logging, progress reporting, and resource access within tools and resources.
- FastMCP supports various transport protocols including SSE, stdio, and WebSockets.
- Sampling allows servers to request completions from client LLMs while maintaining security.
- FastMCP can be installed in Claude Desktop or run as a standalone server.
- Contributions are welcome, with guidelines provided for development and testing.
Llama 2 LLM on DOS
a year ago
- DOS Llama 2 LLM client runs on vintage DOS machines like Thinkpad T42 (2004) and Toshiba Satellite 315CDT (1996).
- The project is open-sourced with executables available on GitHub.
- Porting llama2.c to DOS required overcoming challenges like floating point operations, memory mapping, and DOS filename limits.
- Benchmarks show inference speeds across various vintage and modern systems, with modern systems being faster but some unable to load larger models due to memory issues.
- The project demonstrates the potential of running LLMs on vintage hardware, thanks to the foundational work by Andrej Karpathy.
AI generated text is forbidden with the exception of automated translation
a year ago
- The user expresses skepticism about the correctness of LLM (Large Language Model) outputs.
- Mentions nearly flagging a post for moderator investigation due to suspected AI-generated content.
- Criticizes the summary provided by LLMs as being plausible but not accurate.
- Points out specific inaccuracies in summaries related to app installation options.
- References a community policy statement from April 7 that discourages LLM-generated summaries.
- Raises ethical concerns about contributing LLM output without clear disclosure.
- Clarifies that their opinion does not represent the GrapheneOS project.
12-factor Agents: Patterns of reliable LLM applications
a year ago
- The project '12-factor-agents' is open-source and welcomes contributions.
- Dex has explored various AI agent frameworks and found most 'AI Agents' lack true agentic behavior.
- The article introduces '12-factor agents' as a methodology for building reliable, scalable, and maintainable LLM-powered software.
- Key factors include natural language to tool calls, owning prompts and context window, and making agents stateless reducers.
- The traditional DAG approach is contrasted with agents that let LLMs determine the workflow path dynamically.
- A common journey for SaaS builders involves starting with a framework but often needing to rebuild from scratch for quality.
- The article emphasizes modular concepts from agent building that can be integrated into existing products for faster deployment.
- Disclaimers note the article is not a critique of existing frameworks and acknowledges their contributions to the AI ecosystem.
- The author shares additional resources and links for further reading on effective agent building.
The Impact of MCP and LLMs on Software Development – A Practical Example
a year ago
- Model Context Protocol (MCP) enhances LLMs by enabling them to perform complex tasks in software development with minimal manual intervention.
- MCP allows for one-shot task completion, such as exploring API schemas, writing GraphQL queries, or configuring routers, without deep domain knowledge.
- Scenario 1: MCP helps frontend developers by fetching GraphQL API schemas and verifying queries, streamlining feature implementation.
- Scenario 2: The 'dream query workflow' uses MCP to propose and validate GraphQL schema changes, simplifying API design.
- Scenario 3: MCP aids in configuring complex software like Cosmo Router by leveraging JSON Schema and documentation search.
- MCP is inspired by LSP (Language Server Protocol) and is model and tool agnostic, enhancing various developer tools.
- Security considerations include risks of running MCP servers locally or over SSE, with potential vulnerabilities.
- MCP combined with Agent Mode transforms software development, enabling developers to automate multi-step processes efficiently.
Perplexingly Book-Learned Emacs
a year ago
- Difficulty in finding a programmatic way to track new books by authors.
- Use of Perplexity AI to list books by authors and identify new releases.
- Creation of a simple library, perplexity.el, to interact with Perplexity's API.
- Inconsistency in LLM responses and the challenge of non-repeatable results.
- Manual verification of book listings via bookshop.org and Goodreads due to occasional hallucinations.
- Experimentation with tracking multiple authors and aggregating results for new books.
- Comparison with ISBNdb and its limitations in distinguishing new works from different editions.
- Realization of the potential utility of LLMs despite their inherent jankiness.
Recursive LLM prompts
a year ago
- Implementing recursion using English and an LLM (e.g., GPT-3.5) as the runtime.
- Recursive prompts contain state and update it to approach an end goal (base case).
- Example of an infinitely recursive Fibonacci prompt without a base case.
- Automation via a Python function that calls the OpenAI API recursively.
- Two types of recursion: recursive prompts and recursive Python functions.
- Inspiration from goal-conditioned Reinforcement Learning and General Problem Solver (GPS).
- LLMs use memorized rules (algebraic and atomic) for problem-solving, similar to human cognition.
- Challenges with LLMs generating incorrect facts and potential solutions (e.g., fact-checking, Wikipedia lookups).
- Connections to ACT-R and ReAct (Reasoning + Action) paradigms.
- Exploration of non-tail recursion and stack management within prompts.
- Example of a non-tail recursive prompt for computing factorial(11).
- Instructions for running the recursive prompts using OpenAI Playground or Python scripts.
Why LLM-Powered Programming Is More Mech Suit Than Artificial Human
a year ago
- LLM-powered programming tools like Claude Code act as amplifiers for developers, enhancing their capabilities rather than replacing them.
- These tools require constant human oversight to guide architectural decisions and maintain quality, similar to piloting advanced machinery.
- Vigilance is crucial as AI tools can make bewildering decisions, such as unnecessary code changes or introducing inappropriate dependencies.
- The time cost of writing code has significantly decreased, shifting focus to understanding business problems and designing solutions.
- Developers must develop the skill of ruthlessly discarding code when necessary, leveraging AI's ability to quickly regenerate solutions.
- Experience remains vital in recognizing when AI-generated code may lead to future problems or anti-patterns.
- The 'Centaur Effect' shows that human-AI collaboration outperforms either working alone, combining strategic human direction with AI's computational power.
- Finding the right balance between delegation and control is key to effectively using AI tools in programming.
- The future of programming lies in augmentation, where developers master AI tools to enhance their capabilities and build software more efficiently.
Inner Loop Agents
a year ago
- Inner loop agents allow LLMs to execute tool calls directly without client intervention.
- Regular LLMs rely on clients to parse and execute tools, while inner loop agents handle this internally.
- The LLM emits text with tool calls and a special token (<|eot|>) to signal completion.
- Software like Ollama and vLLM parse LLM output and manage the loop until the <|eot|> token is encountered.
- Inner loop agents enable concurrent tool use during the LLM's thinking process, enhancing efficiency.
- Models like o3 and o4-mini are trained to be agentic, optimizing tool use through reinforcement learning.
- Emergent tool use, where LLMs effectively use new tools without specific training, is still theoretical.
- Current options for tool use include MCP descriptions or training models specifically for tool use.
- Google's Agent 2 Agent (A2A) protocol facilitates communication between different fine-tuned LLM agents.
- Training an LLM with tools doesn't require the tools to be executed on the same host as the LLM.
Gemini 2.5: The First LLM That Understands PDF Layouts
a year ago
- Gemini 2.5 is the first LLM that understands PDF layouts.
- The article was published on April 17, 2025.
- The author invites readers to subscribe to the blog for updates.
- Feedback or questions can be directed to the author.
Meta MCP: Chaining Tools via Prompt-Driven Arguments
a year ago
- Introduces the concept of an MCP tool that chains multiple tools within a single request.
- Explains how arguments for each tool can be dynamically generated using prompts based on previous tool outputs.
- Highlights the utility of this approach for workflows requiring sequential tool calls.
- Provides an example of a multiplexing tool that takes an array of tool call requests.
- Describes the implementation details, including hooks for processing prompt arguments.
- Mentions potential future enhancements like Lua interpretation for conditional tool execution.
- Suggests the possibility of introducing branching logic for more complex workflows.
- Emphasizes the role of the MCP client LLM in orchestrating steps using tools and prompt arguments.
Show HN: Light like the Terminal – Meet GTK LLM Chat Front End
a year ago
- GTK graphical interface for chatting with Large Language Models (LLMs).
- Simple and easy-to-use graphical interface built with GTK.
- Support for multiple conversations in independent windows.
- Integration with python-llm for chatting with various LLM models.
- Modern interface using libadwaita.
- Support for real-time streaming responses.
- Message history with automatic scrolling.
- Markdown rendering of the responses.
- Keyboard shortcuts (Enter to send, Shift+Enter for new line).
- Support for fragments: Include external content (files, URLs, or text snippets) in your prompts.
- Conversation Management: Rename and delete conversations.
- Applet Mode: Run a system tray applet for quick access to recent conversations.
- Model Selection: Choose from different LLM models.
- System Prompt: Set a custom system prompt for each conversation.
- Error Handling: Clear error messages displayed in the chat.
- Dynamic Input: The input area dynamically adjusts its height.
- Keyboard Shortcuts: Enter to send message, Shift+Enter for new line, Ctrl+W to delete the current conversation.
- Installation: pipx install llm, llm install gtk-chat.
- Requirements: llm, Python 3.8 or higher, GTK 4.0, libadwaita, libayatana-appindicator.
- Run the application: llm gtk-applet or llm gtk-chat with optional arguments.
- Development setup: Clone repository, create virtual environment, install dependencies.
- License: GPLv3.
Local LLM inference – impressive but too hard to work with
a year ago
- Local LLM inference is possible but not yet production-ready.
- Compute trends are shifting back towards the edge (local devices) due to benefits like cost, privacy, speed, and offline use.
- Frameworks tested include llama.cpp, Ollama, and WebLLM, with llama.cpp and Ollama showing the best performance.
- Performance metrics show local inference is slower than cloud solutions like OpenAI's GPT-4.0-mini.
- Challenges include finding and deploying the right model for specific tasks and the large size of models leading to slow downloads.
- Future solutions need to simplify training and deploying small, task-specific models and integrate seamlessly with cloud LLMs.
Show HN: Index – new SOTA Open Source browser agent
a year ago
- Index is an open-source browser agent for autonomously executing complex web tasks.
- Powered by reasoning LLMs with vision capabilities, including models like Gemini 2.5 Pro, Claude 3.7 Sonnet, and OpenAI o4-mini.
- Can be installed via pip (`pip install lmnr-index`) and used in projects or via interactive CLI.
- Features include browser state persistence, follow-up messages, real-time streaming updates, and a terminal UI.
- Supports advanced observability via the open-source platform Laminar.
- Available as a serverless API for production use, managing remote browser sessions and infrastructure.
- Example use cases include navigating websites, summarizing content, and creating spreadsheets.
- Supports custom browser configurations like viewport size and connecting to existing Chrome DevTools Protocol endpoints.
- Documentation and API keys setup are required for full functionality.
Paper2Code: Automating Code Generation from Scientific Papers
a year ago
- PaperCoder is a multi-agent LLM framework designed to automate code generation from machine learning papers.
- The framework operates in three stages: planning, analysis, and generation, each handled by specialized agents.
- Planning involves creating a roadmap, system architecture, file dependencies, and configuration files.
- Analysis focuses on interpreting implementation-specific details from the paper.
- Generation produces modular, dependency-aware code based on the analysis.
- PaperCoder was evaluated using model-based and human evaluations, including feedback from original paper authors.
- It outperforms strong baselines in the PaperBench benchmark, demonstrating high-quality, faithful implementations.
- The tool addresses the gap in code availability for machine learning research, aiding reproducibility and further development.
LLMs can see and hear without any training
a year ago
- Official implementation of the paper 'LLMs can see and hear without any training'.
- Install the conda environment using `conda env create -f environment.yml` and activate it with `conda activate MILS`.
- Download datasets: MS-COCO, Clotho, and MSR-VTT, along with annotations and checkpoints.
- Update variables in `paths.py` to set dataset directory and output folder.
- MILS is an inference-only method that can run on a single A100 GPU, but experiments were run on eight A100 GPUs.
- Generate captions for images, audio, and videos using provided scripts and evaluate using corresponding evaluation scripts.
- Generate high-quality images using `main_image_generation_enhancement.py`.
- Perform style transfer by placing style and content images in the `images/` folder and running `main_style_transfer.py`.
- Combine captions from image and audio to create prompts for image generation.
- MILS is available under a CC-by-NC 4.0 license, with third-party content subject to their own licenses.
- Cite the work using the provided BibTeX entry.
How NASA Is Using Graph Technology and LLMs to Build a People Knowledge Graph
a year ago
- NASA is building a People Knowledge Graph using graph databases and LLMs to enhance people analytics.
- The initiative helps identify experts, form teams, and plan future skills by connecting people, projects, and skills.
- Key components include Memgraph for real-time graph queries, AWS cloud infrastructure, and an LLM-powered RAG chatbot.
- Data sources include personnel data, AI/ML projects, and resumes, with skills extracted using Ollama.
- The graph schema includes nodes for employees, projects, skills, and more, labeled as 'Entity' for vector indexing.
- Live demos showcased Cypher queries for expert finding, leadership reports, and project similarity analysis.
- Future plans include scaling to 500K nodes, improving data quality, and expanding the graph with learning goals.
- NASA chose Memgraph over Neo4j for cost efficiency, Python compatibility, and ease of transition.
Show HN: Plan-linter – pre-flight safety checker for AI agent plans
a year ago
- plan-linter is an open-source static analysis toolkit for LLM agent plans.
- It validates machine-readable plans against schemas, policy rules, and heuristics.
- Basic usage involves running `plan-lint` on a JSON plan file.
- Advanced usage allows specifying a policy file for custom validation rules.
- Example policy rules include allowed tools, bounds on arguments, and regex-based token denials.
- Custom rules can be added by creating Python files in the `plan_lint/rules` directory.
- The tool supports CLI and JSON output formats.
- Contributions are welcome, with guidelines for issue discussion, forking, and PR submission.
- Development setup includes creating a virtual environment and installing dependencies.
- Licensed under Apache License 2.0.

first prev1next

About|Login

#llm

Transformer Lab

Teuken-7B-Base and Teuken-7B-Instruct: Towards European LLMs

FastMCP: The fast, Pythonic way to build MCP servers and clients

Llama 2 LLM on DOS

AI generated text is forbidden with the exception of automated translation

12-factor Agents: Patterns of reliable LLM applications

The Impact of MCP and LLMs on Software Development – A Practical Example

Perplexingly Book-Learned Emacs

Recursive LLM prompts

Why LLM-Powered Programming Is More Mech Suit Than Artificial Human

Inner Loop Agents

Gemini 2.5: The First LLM That Understands PDF Layouts

Meta MCP: Chaining Tools via Prompt-Driven Arguments

Show HN: Light like the Terminal – Meet GTK LLM Chat Front End

Local LLM inference – impressive but too hard to work with

Show HN: Index – new SOTA Open Source browser agent

Paper2Code: Automating Code Generation from Scientific Papers

LLMs can see and hear without any training

How NASA Is Using Graph Technology and LLMs to Build a People Knowledge Graph

Show HN: Plan-linter – pre-flight safety checker for AI agent plans