Hasty Briefsbeta

All tags

#machine-learning

166 stories total

Bilingual

Machine Learnability as a Measure of Order in Aperiodic Sequences
7 months ago
- Machine learning models can measure regularity in prime number distributions on Ulam spirals.
- Models trained on higher regions (around 500m) outperform those trained on lower regions (below 25m), indicating more learnable order.
- Classification strategies differ: lower regions focus on identifying primes, while higher regions prioritize eliminating composites.
- Findings align with number theory, suggesting reduced noise and more predictable patterns at higher magnitudes.
- Machine learning could serve as an experimental tool for number theory, especially in studying prime patterns for cryptography.
AMD and Sony's PS6 chipset aims to rethink the current graphics pipeline
7 months ago
- Sony and AMD are teasing new chips for a future console, referred to as 'Project Amethyst'.
- Project Amethyst is a co-engineering effort focused on enhancing machine-learning-based neural networks for better graphics upscaling.
- The project aims to move beyond traditional rasterization techniques, which don't scale well with brute force alone.
- New 'neural arrays' allow compute units to share data, functioning like a single AI engine for more efficient processing.
- This technology will enable more scalable shader engines, allowing larger portions of the screen to be processed at once.
- The goal is to have more on-screen elements enhanced or touched by machine learning (ML).
"Like putting on glasses for the first time": how AI improves quake detection
7 months ago
- A magnitude -0.53 earthquake occurred in Calipatria, California, on January 1, 2008, notable for being very small yet detectable.
- AI and machine-learning tools have automated earthquake detection over the past seven years, surpassing human analysts in accuracy and speed.
- These tools can detect smaller earthquakes, especially in noisy urban environments, providing valuable data on Earth's composition and future hazards.
- Experts compare the adoption of these techniques to 'putting on glasses for the first time,' enhancing clarity in seismic data analysis.
- Machine-learning methods have largely replaced humans in earthquake detection, marking a significant advancement in seismology.
- The potential for AI to impact other areas of seismology, like earthquake forecasting, remains largely untapped.
- Seismologists study how earthquake waves pass through the Earth to infer information about the materials they traverse.
Show HN: Open-source implementation of Stanford's self-learning agent framework
7 months ago
- Agentic Context Engine (ACE) enables AI agents to learn from successes and failures without fine-tuning or training data.
- ACE improves agent performance by 20-35% on complex tasks through continuous learning.
- Key components include Generator (executes tasks), Reflector (analyzes outcomes), and Curator (updates strategies).
- ACE works with 100+ LLM providers like OpenAI, Anthropic, and Google Gemini.
- The Playbook evolves with experience, storing helpful strategies and harmful patterns.
- Demo shows ACE correcting hallucinations (e.g., non-existent seahorse emoji) in real-time.
- Easy installation via pip, with support for LangChain and local models like Ollama.
- Open-source framework based on Stanford & SambaNova research, with community contributions welcome.
Show HN: Syna – Minimal ML and RL Framework Built from Scratch with NumPy
7 months ago
- Syna is a lightweight machine learning framework inspired by DeZero, built using NumPy with a define-by-run approach.
- It includes a basic reinforcement learning framework within a single library, designed for beginners and researchers.
- Performance is not a focus; GPU support is omitted for simplicity and ease of understanding.
- Installation can be done via GitHub or PyPI, with Graphviz required for visualizing computation graphs.
- Example provided for visualizing the computation graph of the fifth derivative of tanh(x).
- Includes a DQN algorithm example for solving CartPole-v1.
- Licensed under the MIT License.
LibCube: Find new sounds from audio synths easier
7 months ago
- The DLL binary release implements 'parameter reduction' machine learning code for audio synthesizers.
- Reduces synthesizer parameters to 2-3 dimensions for easier sound search using machine learning.
- Includes a C API library ('cube_interface.h') and DLL ('cube_param.dll') for integration.
- Uses t-SNE algorithm for dimension reduction, scaling O(N*log(N)) with presets.
- Supports multi-core CPUs for faster computations.
- Example program can be compiled with MSYS2 MINGW or Visual C++ in Windows.
- Depends on Linux DLLs with licenses that allow commercial use.
- Parameter reconstruction uses neural networks, requiring significant CPU time.
- Parameters can be changed offline but not in real-time automation.
- Includes example sounds and a YouTube demo with Sylent1 VST2 synthesizer.
- Author welcomes feedback and integration into audio synthesizers.
- Limited support available; contact [email protected] for questions.
LightlyStudio – an open-source multimodal data curation and labeling tool
6 months ago
- LightlyStudio is an open-source tool for data curation, annotation, and management.
- Built with Rust for performance, it supports COCO and ImageNet datasets on a Macbook Pro with M1 and 16GB RAM.
- Compatible with Python 3.8+ on Windows, Linux, and MacOS.
- Install via pip: `pip install lightly-studio`.
- Example datasets can be downloaded from a GitHub repository or use your own YOLO/COCO dataset.
- Includes examples for image-only datasets, YOLO object detection, COCO instance segmentation, and COCO captions.
- LightlyStudio features a powerful Python interface for dataset indexing, querying, and manipulation.
- Supports loading data from cloud storage (e.g., S3, GCS) and local folders.
- Sample attributes include ID, file name, path, tags, and metadata, which can be accessed and modified.
- Dataset queries allow filtering, sorting, and slicing operations using expressions.
- Premium feature for automated data selection to pick the most useful samples based on typicality and diversity.
- Version 0.4.0 released as a preview on 2025-10-21.
- Contributions are welcome via the issues page for tasks and improvements.
The Continual Learning Problem
6 months ago
- Memory layers are proposed as a solution for continual learning, enabling models to update parameters without catastrophic forgetting.
- Memory layers allow for high-capacity, sparse updates, with only a small subset of parameters active during each forward pass.
- Experiments show memory layers reduce forgetting significantly compared to full finetuning and LoRA, with only an 11% performance drop on NaturalQuestions versus 89% and 71% respectively.
- Continual learning is framed as two subproblems: generalization (learning important bits from data) and integration (avoiding forgetting while updating knowledge).
- Memory layers use sparse attention over a pool of learned keys and values, enabling targeted updates and efficient information storage.
- Sparse memory finetuning leverages TF-IDF-like ranking to selectively update memory slots, minimizing interference with existing knowledge.
- Memory architectures offer potential advantages for larger models, with opportunities for better interpretability and organization of knowledge.
- Optimizer choice (e.g., SGD vs. AdamW) impacts continual learning performance, with SGD showing better results for sparse memory finetuning.
- Future directions include scaling memory layers to larger models, improving benchmarks, and exploring hybrid approaches between memory layers and in-context learning.
The State of Machine Learning Frameworks in 2019
6 months ago
- Deep learning frameworks have evolved significantly since 2012, with PyTorch and TensorFlow emerging as the main contenders.
- PyTorch has gained dominance in research, with a majority of papers at top conferences now using it, while TensorFlow remains popular in industry.
- Researchers prefer PyTorch for its simplicity, great API, and performance, despite TensorFlow's historical advantages in production environments.
- TensorFlow 2.0 introduced eager mode to compete with PyTorch's ease of use, but challenges remain in seamless deployment and performance.
- PyTorch introduced TorchScript for production needs, but TensorFlow still leads in industry due to better deployment tools like TensorFlow Lite and Serving.
- The future of ML frameworks depends on how well PyTorch can address production needs and how TensorFlow can regain researcher interest.
- Emerging frameworks like Jax offer new capabilities, such as higher-order differentiation, which could influence future research directions.
- The battle between PyTorch and TensorFlow may become irrelevant as new computing models and hardware paradigms emerge.
- Machine learning frameworks shape research by enabling or restricting the ideas researchers can explore easily.
- The community's focus remains on advancing ML research and democratizing AI, regardless of framework preferences.
Generalized K-Means Clustering
6 months ago
- The project follows security best practices with automated dependency updates via dependabot.yml.
- Version 0.6.0 introduces a modern, RDD-free DataFrame-native API with Spark ML integration.
- Generalizes K-Means to multiple Bregman divergences and advanced variants like Bisecting, X-Means, Soft/Fuzzy, Streaming, K-Medians, and K-Medoids.
- Supports multiple divergences: Squared Euclidean, KL, Itakura–Saito, L1/Manhattan, Generalized-I, and Logistic-loss.
- Tested on tens of millions of points in 700+ dimensions.
- Comprehensive CI pipeline ensures quality across multiple dimensions including lint, build matrix, test matrix, and security scanning.
- Provides detailed diagnostics for tuning performance and avoiding OOM errors.
- Automatic validation at fit time for different divergences with actionable error messages.
- Models implement DefaultParamsWritable/Readable for persistence across Spark versions.
- Legacy RDD API is kept for backward compatibility but new development should use the DataFrame API.
Show HN: Create-LLM – Train your own LLM in 60 seconds
6 months ago
- create-llm is a tool for quickly setting up custom language model training projects.
- It offers four templates: NANO (1M params), TINY (6M params), SMALL (100M params), and BASE (1B params).
- The tool provides everything needed for training, including model architecture, data preprocessing, tokenizer training, and deployment tools.
- It includes features like auto-detection of vocab size, warnings for model/data mismatches, and suggestions for optimal hyperparameters.
- Optional integrations with WandB, HuggingFace, and SynthexAI are available.
- The tool is designed for various use cases, from learning and prototyping to production and research.
- It supports real-time training monitoring, model comparison, and automatic checkpoint management.
- Minimum data requirements and data quality tips are provided to avoid overfitting.
- Deployment options include Hugging Face Hub, Replicate, Docker, and cloud platforms.
- Common issues and solutions are documented, such as vocab size mismatch and CUDA out of memory errors.
- The tool requires Node.js, npm, Python, and PyTorch, with specific hardware recommendations for each template.
- Contributions are welcome, with areas ranging from bug fixes to new features and integrations.
- Future plans include more model architectures, distributed training support, and advanced optimization techniques.
Sieve (YC X25) Is Hiring Engineers to build video datasets for frontier AI
6 months ago
- 500K hours of high-quality, diverse video clips available for AI applications.
- Video data sourced from public, private, and synthetic sources.
- Purpose-built video understanding models with human QA ensure high-quality, training-ready data.
- Thousands of petabytes of video data in a growing library.
- New data formats (paired, time-synced, conversational) to enhance model capabilities.
- Pre-packaged datasets available for exploration and purchase.
- Fast delivery (1-2 days) via storage bucket access.
- Scalable API designed to process millions of hours of video.
- Compliant data with specific filtering and licensing options.
- Dedicated partnerships to tailor data solutions for research teams.
- Secure data handling with encryption, custom retention, and SOC 2 Type 2 compliance.
On-Policy Distillation
6 months ago
- LLMs achieve expert performance through a combination of pre-training, mid-training, and post-training stages.
- Smaller models with specialized training can outperform larger generalist models in specific domains.
- On-policy training involves sampling from the student model and assigning rewards, while off-policy training relies on external target outputs.
- On-policy distillation combines the relevance of RL with the dense reward signal of distillation, grading each token of the student's trajectory.
- On-policy distillation is shown to be more compute-efficient than RL, achieving similar performance with fewer steps.
- Distillation can effectively reuse training data, allowing multiple training epochs on the same prompt without overfitting.
- On-policy distillation is useful for continual learning, enabling models to acquire new knowledge without degrading prior capabilities.
- The method is applied to tasks like mathematical reasoning and personalized assistant training, demonstrating its versatility.
Kimi Linear: An Expressive, Efficient Attention Architecture
6 months ago
- Kimi Linear achieves 51.0 performance on MMLU-Pro with similar speed as full attention.
- On RULER (128k context length), it shows Pareto-optimal performance (84.3) and a 3.98x speedup.
- Kimi Linear is 6.3x faster TPOT compared to MLA, especially for long sequences (1M tokens).
- Kimi Delta Attention (KDA) is a refined version of Gated DeltaNet with an efficient gating mechanism.
- Reduces KV caches by up to 75% and boosts decoding throughput significantly.
- Open-sourced KDA kernel in FLA and released two model checkpoints trained with 5.7T tokens.
- Hybrid architecture with a 3:1 KDA-to-global MLA ratio reduces memory usage while maintaining quality.
- Superior performance in long-context and RL-style benchmarks on 1.4T token training runs.
- Achieves up to 6x faster decoding and reduces time per output token (TPOT).
- Example code provided for using the Kimi Linear model with Python and deployment via vllm.
Gt: [experimental] multiplexing tensor framework
6 months ago
- GT is an experimental multiplexing tensor framework for distributed GPU computing.
- It rejects the clunky lock-step paradigm used in ML research, embracing dynamic scheduling and asynchronous execution.
- GT consists of three components: clients (users), dispatcher (coordinator), and workers (one per GPU).
- Clients emit pure functional instructions, which the dispatcher rewrites to be GPU-aware and sends to workers.
- Workers asynchronously process instructions, optionally JIT compiling.
- Instruction streams are annotated with signals for sharding and hot paths for JIT hints.
- YAML configs supplement annotations for sharding and compilation, but annotations can be safely ignored.
- GT automatically spins up an asynchronous dispatching server and GPU worker in the background.
- Features include high-performance transport (ZeroMQ), autograd support, PyTorch-compatible API, and signal-based sharding.
- Additional features: real-time monitoring, instruction logging, AI-assisted development, and comprehensive documentation.
- Installation is via pip, and usage includes auto-server mode, tensor operations, autograd, and signal-based sharding.
- Examples demonstrate basic tensor operations, signal-based sharding, compilation directives, debug utilities, and visualization.
- GT is designed for simplicity, readability, and collaboration with AI coding assistants.
- Contributions are welcome, with detailed guidelines provided.
- Licensed under MIT.
Pre-training under infinite compute
6 months ago
- Pre-training under infinite compute explores optimizing language model pre-training with fixed data and no compute constraints.
- Existing data-constrained approaches like increasing epoch count and parameter count eventually overfit.
- Optimal weight decay is found to be 30× larger than standard practice, improving regularization.
- Ensembling independently trained models achieves a lower loss asymptote than regularized recipes.
- Combining epoching, regularization, parameter scaling, and ensemble scaling achieves a 5.17× data efficiency improvement.
- Distilling an ensemble into a smaller student model retains 83% of the ensembling benefit.
- Interventions generalize to downstream benchmarks, showing 9% improvement in pre-training evals and 17.5× data efficiency in math tasks.
- Simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future.
Python library for quantum computing, quantum ML, and quantum chemistry
6 months ago
- PennyLane is a Python library for quantum computing, machine learning, and chemistry.
- It supports building quantum circuits, running on simulators or hardware, and includes features like mid-circuit measurements.
- Offers tools for mastering quantum algorithms, from NISQ to fault-tolerant computing.
- Integrates with machine learning frameworks like PyTorch, TensorFlow, JAX, Keras, and NumPy for hybrid models.
- Provides access to quantum datasets to accelerate research and algorithm development.
- Supports just-in-time compilation and advanced features like adaptive circuits and real-time feedback.
- Requires Python 3.11+ and can be installed via pip or Docker.
- Includes resources like demos, coding challenges, FAQs, and documentation for learning and development.
- Encourages contributions, bug reports, and feature suggestions, with contributors listed in releases and papers.
- Community engagement through discussion forums and adherence to a Code of Conduct.
- Free and open-source under the Apache License 2.0, with research encouraged to cite the PennyLane paper.
Show HN: TabPFN-2.5 – SOTA foundation model for tabular data
6 months ago
- TabPFN-2.5 is the next generation tabular foundation model, scaling to 20× more data cells than TabPFNv2.
- It outperforms tuned tree-based models and matches AutoGluon 1.4's accuracy on benchmarks with up to 50,000 data points and 2,000 features.
- A new distillation engine converts TabPFN-2.5 into compact MLP or tree ensembles, reducing latency while maintaining accuracy.
- Tabular foundation models (TFMs) like TabPFN-2.5 offer training-free prediction with strong calibration and generalization, eliminating the need for extensive tuning.
- TabPFN-2.5 supports messy, heterogeneous data, including categorical features, missing values, and outliers, scaling up to 50,000 samples and 2,000 features.
- The model significantly improves inference latency and deployment flexibility through distillation.
Pose Animator – An open source tool to animate SVG characters via motion capture
6 months ago
- Pose Animator is an open-source web animation tool that brings SVG characters to life using body detection from a webcam.
- It utilizes TensorFlow.js models PoseNet and FaceMesh for real-time human perception in the browser.
- The tool applies skeletal animation concepts to vector characters, deforming SVG paths based on detected keypoints.
- Pose Animator uses Linear Blend Skinning (LBS) to deform vector paths, with special handling for Bezier curves.
- The rig structure includes 90 keypoints (73 from FaceMesh, 17 from PoseNet) and 78 bones for full-body animation.
- Motion stabilization techniques are employed to reduce jitter from model predictions using confidence scores.
- Future improvements may include better mesh rigging, weight painting tools, and support for raster images in SVGs.
- Users can try live demos, create custom SVG characters, and share creations using #PoseAnimator.
Reverse Engineering a Neural Network's Clever Solution to Binary Addition (2023)
6 months ago
- Small neural networks (<1000 parameters) can be surprisingly effective for specialized tasks.
- A neural network was trained to perform 8-bit binary addition, including handling overflow.
- The network successfully learned the task with as few as 422 parameters across 3 layers.
- The network used a custom activation function (Ameo) in the first layer and tanh in others.
- The network's solution resembled a digital-to-analog converter (DAC), converting binary inputs to analog signals.
- Neurons in the first layer generated sine wave-like outputs with periods matching binary digit switches.
- Later layers combined and routed these signals, making them more square wave-like.
- The network's approach was unexpected, leveraging analog signal processing rather than digital logic gates.
- The findings suggest that more efficient architectures could reduce the size of large neural networks.
- The experiment highlights the power of gradient descent and optimization algorithms in neural networks.

first prev4next

About|Login

#machine-learning

Machine Learnability as a Measure of Order in Aperiodic Sequences

AMD and Sony's PS6 chipset aims to rethink the current graphics pipeline

"Like putting on glasses for the first time": how AI improves quake detection

Show HN: Open-source implementation of Stanford's self-learning agent framework

Show HN: Syna – Minimal ML and RL Framework Built from Scratch with NumPy

LibCube: Find new sounds from audio synths easier

LightlyStudio – an open-source multimodal data curation and labeling tool

The Continual Learning Problem

The State of Machine Learning Frameworks in 2019

Generalized K-Means Clustering

Show HN: Create-LLM – Train your own LLM in 60 seconds

Sieve (YC X25) Is Hiring Engineers to build video datasets for frontier AI

On-Policy Distillation

Kimi Linear: An Expressive, Efficient Attention Architecture

Gt: [experimental] multiplexing tensor framework

Pre-training under infinite compute

Python library for quantum computing, quantum ML, and quantum chemistry

Show HN: TabPFN-2.5 – SOTA foundation model for tabular data

Pose Animator – An open source tool to animate SVG characters via motion capture

Reverse Engineering a Neural Network's Clever Solution to Binary Addition (2023)