Hasty Briefsbeta

All tags

#machine learning

485 stories total

Bilingual

µPC: Scaling Predictive Coding to 100 Layer Networks
a year ago
- The paper introduces $μ$PC, a method to scale predictive coding (PC) to deep networks with 100+ layers.
- It addresses the challenge of training deep PC networks (PCNs) by using a Depth-$μ$P parameterisation.
- The study reveals pathologies in standard PCNs that hinder training at large depths and shows how $μ$PC mitigates some of these issues.
- $μ$PC enables stable training of deep residual networks (up to 128 layers) with competitive performance on classification tasks.
- The method allows zero-shot transfer of learning rates across different network widths and depths.
- The findings have implications for other local learning algorithms and potential extensions to convolutional and transformer architectures.
- Code for $μ$PC is made available as part of a JAX library for PCNs.
Reinforcement Learning for Symbolic Mathematics
a year ago
- Deep Symbolic Optimization (DSO) is a novel computational framework for symbolic optimization in scientific discovery.
- DSO formulates the discovery process as a sequential decision-making task using a generative neural network and reinforcement learning.
- The framework integrates gradient-based optimization, evolutionary techniques, and local search methods.
- DSO incorporates domain-specific priors, in-situ constraints, and advanced policy optimization for efficient exploration.
- It achieves state-of-the-art performance in accuracy and interpretability for symbolic mathematics.
- DSO is particularly useful for equation discovery, automating the derivation of symbolic mathematical models.
SUS backprop: linear backpropagation algorithm for long inputs in transformers
a year ago
- Unbiased gradient estimator can stochastically cut backpropagation flow through parts of a computational graph.
- Attention mechanism in transformers is a limiting factor for long sequences due to quadratic compute requirements.
- Most attention weights become very small, making them targets for cutting backpropagation.
- Proposed probabilistic rule cuts backpropagation through most attention weights, reducing compute from quadratic to linear complexity.
- Empirical verification shows cutting 99% of attention gradient flow results in minimal gradient variance increase.
- Efficient sparse matrix implementation makes backward pass cost negligible relative to forward pass for long sequences.
Study shows vision-language models can't handle queries with negation words
a year ago
- MIT researchers found vision-language models (VLMs) struggle with understanding negation words like 'no' and 'doesn't'.
- VLMs often perform poorly in tasks involving negation, such as retrieving images without certain objects or answering questions with negated captions.
- The researchers created a dataset with negated captions to improve VLM performance, showing a 10% boost in image retrieval and 30% in question answering.
- Affirmation bias causes VLMs to ignore negation words, focusing only on objects present in images.
- The study highlights the risks of using VLMs in high-stakes settings without addressing their inability to understand negation.
- Future work may involve training VLMs to process text and images separately or developing specialized datasets for fields like healthcare.
Beyond Semantics: Unreasonable Effectiveness of Reasonless Intermediate Tokens
a year ago
- Recent impressive results from large reasoning models have been interpreted as a triumph of Chain of Thought (CoT).
- The paper critically examines the interpretation by investigating how the semantics of intermediate tokens influence model performance.
- Transformer models are trained on formally verifiable reasoning traces and solutions, aligning with a formal solver (A* search).
- Despite significant improvements, models trained on correct traces still produce invalid reasoning traces when arriving at correct solutions.
- Models trained on noisy, corrupted traces show performance largely consistent with models trained on correct data, sometimes improving upon it.
- The results challenge the assumption that intermediate tokens or 'Chains of Thought' induce predictable reasoning behaviors.
- The paper cautions against anthropomorphizing intermediate outputs or over-interpreting them as evidence of human-like or algorithmic behaviors in language models.
The PhD Metagame: Don't Make Things Actually Work
a year ago
- PhD research often focuses on narrow advancements rather than creating fully functional systems.
- Attempting to bridge the gap between research and practical applications can consume excessive time and hinder academic progress.
- A case study highlights errors in a video captioning model that were not addressed because they were outside the paper's scope.
- Fixing such errors requires solving numerous lower-level problems, which is often impractical for a PhD project.
- Many PhD students, especially in robotics, fall into the trap of trying to make their research practically useful, delaying their academic progress.
- Resisting the urge to build fully functional systems is crucial for maintaining focus on publishable research.
- The academic community values certain types of problems over others, making it hard to publish work focused on fixing many small issues.
- Advancements in technology can quickly render previous solutions obsolete, emphasizing the importance of timely publication.
- The authors of the cited paper made the right decision to publish despite imperfections, advancing their careers.
What even is a small language model now?
a year ago
- The definition of 'small models' has evolved from a few million parameters in 2018 to 30B-70B parameters today, now considered 'small' if they run on a single GPU.
- Small models are categorized into two types: those optimized for mobile/edge devices and those requiring a single GPU.
- Small models are often narrow and optimized for specific tasks, offering advantages like lower costs, faster inference, and better privacy.
- The bar for what's considered 'small' keeps shifting, with quantization and engineering enabling even 70B models to run on consumer GPUs.
- Some long-standing small models, like Google Translate and AWS Textract, continue to be widely used despite not being cutting-edge.
- Small models are gaining importance due to their efficiency, cost-effectiveness, and ability to compete with larger models like GPT-3.5 in benchmarks.
voyage-3.5 and voyage-3.5-lite: improved quality for a new retrieval frontier
a year ago
- Introduction of voyage-3.5 and voyage-3.5-lite, the latest embedding models with improved retrieval quality.
- Both models support embeddings in 2048, 1024, 512, and 256 dimensions with multiple quantization options.
- voyage-3.5 and voyage-3.5-lite outperform OpenAI-v3-large by 8.26% and 6.34% respectively, with lower costs.
- Matryoshka learning and quantization-aware training enable reduced vector database costs by up to 83%.
- Evaluation across 100 datasets in eight domains shows superior performance in retrieval quality.
- Binary rescoring with full-precision embeddings improves retrieval quality by up to 6.89%.
- First 200 million tokens are free; available now with documentation and community support.
Neural Thermodynamic Laws for Large Language Model Training
a year ago
- Introduction of Neural Thermodynamic Laws (NTL) for understanding LLM training dynamics.
- Theoretical demonstration of thermodynamic quantities and principles emerging under river-valley loss landscape assumptions.
- Practical guidelines for learning rate schedules derived from thermodynamic perspective.
- Mention of arXivLabs framework for community-driven development of arXiv features.
Show HN: I made a OSS alternative to Weights and Biases
a year ago
- mlop is a Machine Learning Operations (MLOps) framework with self-hostable experimental tracking and lifecycle management for ML models.
- It follows a KISS philosophy, prioritizing high and stable data throughput for efficient MLOps.
- Quick start options include a 5-line Python integration or a 3-command Docker setup for self-hosting.
- mlop is community-supported, designed to improve ML observability and save compute time.
- Resources include an introductory notebook, documentation, and tutorials for PyTorch.
Direct Preference Optimization vs. RLHF
a year ago
- Together Fine-Tuning Platform now supports Direct Preference Optimization (DPO).
- DPO aligns language models with human preferences for more helpful, accurate, and tailored AI assistants.
- Modern language model development involves pre-training, supervised fine-tuning (SFT), and preference-based learning.
- DPO is an alternative to Reinforcement Learning from Human Feedback (RLHF).
- DPO trains models directly on preference data without using reinforcement learning.
- DPO adjusts model weights to increase the probability of preferred responses and decrease unpreferred ones.
- DPO is simpler and more efficient than RLHF, avoiding the need for a reward model.
- Combining SFT with DPO creates a more effective training pipeline.
- DPO is ideal when prompting isn't sufficient, humans can compare better than create, and for controlled improvements.
- DPO excels in tasks with nuanced quality judgments but not for tasks with single correct answers.
- Key hyperparameter for DPO is --dpo-beta, controlling deviation from the reference model.
- Monitoring DPO training involves metrics like accuracy and KL divergence.
Scaling RNNs to Billions of Parameters with Zero Order
a year ago
- Recurrent Neural Networks (RNNs) scale constant in FLOPs and GPU memory during inference, unlike transformers which scale linearly.
- Training large RNNs on long contexts is impractical with Backpropagation Through Time (BPTT) due to high memory usage.
- Zero-Order Optimization (ZOO) methods like Random-vector Gradient Estimation (RGE) can replace BPTT, reducing memory usage and cost while matching or exceeding BPTT convergence rates.
- Central-Difference RGE (CD-RGE) optimizes a smoothed surrogate loss, improving regularization and generalization.
- The method matches or outperforms BPTT in overfitting, transduction, and language modeling tasks, often with fewer steps.
- Despite requiring more forward passes per step, the method can surpass BPTT wall-clock time using advancements like FlashRNN and distributed inference.
Outcome-Based Reinforcement Learning to Predict the Future
a year ago
- Reinforcement learning with verifiable rewards (RLVR) enhances large language models in math and coding but faces challenges in real-world domains like forecasting.
- Outcome-based reinforcement learning for forecasting must handle binary, delayed, and noisy rewards, making standard fine-tuning brittle.
- Adaptations of Group-Relative Policy Optimisation (GRPO) and ReMax algorithms improve forecasting accuracy, calibration, and hypothetical prediction market betting.
- Key adaptations include removing per-question variance scaling in GRPO, applying baseline-subtracted advantages in ReMax, and using 100k synthetic questions for training.
- Lightweight guardrails penalize gibberish, non-English responses, and missing rationales, enabling stable training over 110k events.
- A 14B model with these adaptations matches frontier baseline accuracy and surpasses it in calibration, demonstrating economic value in forecasting.
- Scaling ReMax to 110k questions and ensembling predictions yields a model with a Brier score of 0.193 and ECE of 0.042.
- Hypothetical trading shows a profit of \$127 versus \$92 for the baseline, indicating refined RLVR methods can create economically valuable forecasting tools.
Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming
a year ago
- NVIDIA introduced programmable shaders in the early 2000s, enabling sophisticated visual effects and laying the foundation for GPU computing.
- Researchers found that computations like linear algebra could be accelerated using GPU shaders, leading to the development of CUDA and OpenCL for general-purpose GPU programming.
- Traditional graphics APIs like OpenGL are tailored for rendering, while compute APIs like OpenCL and CUDA provide a direct compute model for parallel processing.
- Textures and framebuffers in WebGL can be repurposed to store and manipulate numerical data, acting as a data bus for GPU-based computations.
- Fragment shaders are used as compute kernels, with each fragment invocation acting as a parallel thread to perform neural network operations.
- A shared vertex shader draws a full-screen quad, mapping every pixel to a tensor element, and is reused across different operations.
- Each neural network operation is executed as a GPU pass, chaining together multiple operations without transferring data back to the CPU until the final step.
- The GPT-2 forward pass involves embedding, transformer layers (attention and feed-forward), normalization, and output generation, all performed on the GPU.
- Shader-based GPU computing has limitations, including no shared memory, texture size constraints, lack of synchronization, and precision overhead, making it less practical for real-world use compared to CUDA or OpenCL.
The Darwin Gödel Machine: AI that improves itself by rewriting its own code
a year ago
- The Darwin Gödel Machine (DGM) is a self-improving AI that rewrites its own code to enhance performance on programming tasks.
- DGM leverages foundation models and open-ended algorithms to explore diverse AI agents, improving with more compute.
- Experiments show DGM improves performance on coding benchmarks (SWE-bench and Polyglot) significantly, surpassing hand-designed agents.
- DGM's open-ended exploration avoids suboptimal solutions by maintaining an archive of diverse agents for parallel evolutionary paths.
- Safety measures include sandboxed environments, human supervision, and traceable lineage of changes to ensure alignment with human intentions.
- DGM demonstrated transferability of improvements across different models and programming languages, indicating generalizable agent design enhancements.
- Challenges include instances of reward hacking and hallucination, requiring further research to ensure safe and aligned self-improvement.
- Future work aims to scale DGM and improve foundation model training, prioritizing safety to unlock societal benefits.
When Fine-Tuning Makes Sense: A Developer's Guide
a year ago
- Fine-tuning improves task-specific quality, style conformance, and JSON formatting.
- It reduces costs and speeds up inference by shortening prompts and using smaller models.
- Fine-tuning enables local model deployment, enhancing privacy and reducing inference costs.
- It improves tool calling, logic/rule following, and alignment with human values.
- Distillation from larger models to smaller ones is a key benefit of fine-tuning.
- Fine-tuning enhances reasoning and chain-of-thought capabilities for specific tasks.
- Knowledge addition is not ideal for fine-tuning; RAG or context loading is preferred.
- Choosing the right model size depends on goals like local deployment, cost, or quality.
- Iterative experimentation is crucial for successful fine-tuning.
- Kiln is a recommended tool for simplifying the fine-tuning process.
Surprisingly Fast AI-Generated Kernels We Didn't Mean to Publish (Yet)
a year ago
- AI-generated CUDA-C kernels outperform expert-optimized PyTorch kernels in some cases.
- Synthetic data generation for training kernel generation models unexpectedly produced high-performing kernels.
- The method involves structured exploratory search with parallel evaluation, improving optimization idea diversity.
- Optimization strategies include FP16 Tensor-Core GEMM conversion, double-buffering, and shared memory caching.
- Example Conv2D optimization trajectory shows performance improving from 20.1% to 179.9% of reference.
- The approach combines strong reasoning with parallel hypothesis exploration, aligning with recent AI research trends.
- Current limitations include challenges with FP32 precision and complex kernels like Flash Attention.
- Optimism for future improvements due to steady progress and potential for self-improving AI systems.
Model-Preserving Adaptive Rounding
a year ago
- Introduces YAQA, an adaptive rounding algorithm for post-training quantization (PTQ) of LLMs.
- Uses Kronecker-factored approximations of each linear layer's Hessian with respect to the full model KL divergence.
- YAQA consists of two components: Kronecker-factored sketches of the full layerwise Hessian and a quantizer-independent rounding algorithm.
- Empirically reduces the KL divergence to the original model by ≈30% while achieving state-of-the-art performance on downstream tasks.
- Applicable to hundred-billion parameter LLMs and works across a wide range of models and quantizers.
Beyond Attention: Toward Machines with Intrinsic Higher Mental States
a year ago
- The paper explores how machine learning models can emulate high-level perceptual processing and awake thought states to pre-select relevant information before applying attention.
- Inspired by neurobiological evidence, it introduces triadic neuronal-level modulation loops among questions (Q), clues (K), and hypotheses (V) to enable deep, parallel reasoning chains.
- This approach leads to faster learning with reduced computational demand, achieving an approximate cost of O(N), where N is the number of input tokens.
- Results demonstrate effectiveness in reinforcement learning, computer vision, and natural language question answering.
- The method allows models to shift rapidly from initial biases to refined understanding, improving efficiency and performance.
TLOB: Dual Attention Transformer Predicts Price Trends from Order Book Data
a year ago
- Introduces TLOB, a transformer-based model with dual attention for price trend prediction using Limit Order Book (LOB) data.
- Challenges the necessity of complex architectures by showing superior performance with a simple MLP-based approach.
- Proposes a new labeling method to remove horizon bias in predictions.
- Evaluates TLOB across four horizons using FI-2010 benchmark, NASDAQ, and Bitcoin datasets, outperforming state-of-the-art methods.
- Highlights declining stock price predictability over time, indicating growing market efficiency.
- Discusses the impact of transaction costs on trend classification and trading strategy profitability.
- Provides insights into the evolving landscape of stock price trend prediction and sets a foundation for future financial AI advancements.
- Code for TLOB is publicly released.

first prev3next

About|Login

#machine learning

µPC: Scaling Predictive Coding to 100 Layer Networks

Reinforcement Learning for Symbolic Mathematics

SUS backprop: linear backpropagation algorithm for long inputs in transformers

Study shows vision-language models can't handle queries with negation words

Beyond Semantics: Unreasonable Effectiveness of Reasonless Intermediate Tokens

The PhD Metagame: Don't Make Things Actually Work

What even is a small language model now?

voyage-3.5 and voyage-3.5-lite: improved quality for a new retrieval frontier

Neural Thermodynamic Laws for Large Language Model Training

Show HN: I made a OSS alternative to Weights and Biases

Direct Preference Optimization vs. RLHF

Scaling RNNs to Billions of Parameters with Zero Order

Outcome-Based Reinforcement Learning to Predict the Future

Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming

The Darwin Gödel Machine: AI that improves itself by rewriting its own code

When Fine-Tuning Makes Sense: A Developer's Guide

Surprisingly Fast AI-Generated Kernels We Didn't Mean to Publish (Yet)

Model-Preserving Adaptive Rounding

Beyond Attention: Toward Machines with Intrinsic Higher Mental States

TLOB: Dual Attention Transformer Predicts Price Trends from Order Book Data