Hasty Briefsbeta

All tags

machine learning

23 stories total

Bilingual

Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators
2 months ago
- AI pricing is based on tokens (roughly 3/4 of a word), with separate charges for input and output; output is typically 3-5x more expensive than input.
- Prompt caching can reduce repeated input costs by up to 90%, and batch processing offers ~50% discounts, which many cost estimates ignore.
- A live table shows pricing for 232+ flagship models, including GPT-5.x, Claude 5 variants, DeepSeek V4, Gemini 3.x, Grok 4.x, and GLM 5.2, with costs ranging from $0.09 to $10.00 per million input tokens.
- Calculators are provided to convert token prices into real monthly costs for chatbots, agents, or API workloads.
Autoregressive Language Model on the 6502 Processor
2 months ago
- The project runs a language model on a BBC Micro (6502 CPU) with only 25KB of user memory (9KB code, 13KB weights).
- The model uses BitNet ternary quantization (-1,0,1) packing 4 parameters per byte, storing 52k parameters in 13KB.
- A Mamba (SSM) architecture is chosen over attention or GRU because it requires fixed-size state and avoids instability issues on the 6502.
- Inference is coded in C (CC65) and transferred to the BBC Micro via a custom audio cable and PlayUEF.
- The vocabulary is limited to 26 letters plus space to save parameters on embedding/decoder layers.
- Token sampling uses a lookup table for softmax (exponential) and a fixed-seed pseudo-random generator.
- The model runs on real hardware but is not intelligent; it demonstrates the feasibility of running modern ML on 1980s hardware.
- The work highlights mechanical sympathy: designing models with hardware constraints in mind.
Explorative Modeling – A new axis for pre-training
2 months ago
- Deep learning revolutionized AI with end-to-end training, but generative models still avoid this approach due to the difficulty of handling multi-modal distributions.
- Explorative Modeling (XM) factors the training loop by exploring K candidate matches between model generations and data, training on the best to commit to modes instead of blurring them.
- Increasing exploration adds a third pretraining axis beyond parameters and data, improving performance across images, video, and language as scale increases.
- Gains from exploration grow with scale: from 7% to 36% with data scaling and from 13% to 23% with model scaling; efficiency improves up to 4.1x in FLOP, 6.2x in samples, and 47% in parameters.
- XM achieves a near-state-of-the-art 1.43 FID on ImageNet without guidance and enables end-to-end reconstructive modeling, matching diffusion on control tasks with 16-256x fewer inference steps.
"Uncensored" open LLMs are measurably more optimistic than their base models
2 months ago
- Abliteration of refusal direction in model weights leads to measurable side effects beyond refusal removal, as shown by systematic behavioral shifts in a decision-making task that elicits no refusals.
- Abliterated models from two Mixture-of-Experts families (Gemma-4-26B-A4B-it and Qwen3-30B-A3B-instruct) showed increased optimism (higher positivity in decisions), longer justifications, and fewer uncertainty words in self-critiques.
- The effect on expressed confidence was opposite across families: Gemma models became less confident, while Qwen models became more confident.
- The ablation process did not degrade instruction-following ability, and neither abliterated nor base models demonstrated economic skill in stock predictions.
- Two contamination channels were detected in community-modified checkpoints, suggesting toolchain artifacts are common in such studies.
- Deploying an 'uncensored' model as an agent means deploying a measurably different decision-maker, not the base model without refusals.
Overtraining as the path to human-like AI
2 months ago
- Gwern proposes that overtraining, or 'grokking,' large neural networks on small datasets could lead to human-like AI by forcing deeper generalization.
- Current LLMs fail to generalize as well as humans because they are trained on massive data with minimal overtraining, unlike the grokking process.
- Grokking involves a sudden capability jump after prolonged training, as models shift from memorization to understanding underlying rules.
- Gwern suggests training a 100-trillion-parameter model on a constrained dataset to induce grokking, opposite to current AI lab practices.
- Political and technical obstacles, including high costs and apparent stagnation, may prevent labs from attempting this risky approach.
- The article argues that grokking, not pure scaling or reasoning, might be a viable path to artificial superintelligence.
Practical Simulations for Machine Learning is out now
2 months ago
- New book co-authored with Mars, Tim, and Jon covering Unity ML-Agents simulations.
- Explains how to use simulations to train machine learning models.
- Available in bookshops and on O'Reilly Online Learning platform.
Hello Deep Learning - Bert Hubert's writings
2 months ago
- The tutorial provides a GPU-free, from-scratch introduction to modern machine learning.
- It aims to explain concepts deeply without relying on high-level frameworks like PyTorch.
- The tutorial avoids pure mathematical approaches that may only suit seasoned mathematicians.
- The goal is to develop functional neural networks from scratch that yield impressive results on real-life data.
- The tutorial includes code and Markdown files hosted on GitHub for collaborative improvements.
- It covers 12 chapters, starting from linear combinations to advanced topics like attention and transformers, including practical applications like handwritten digit recognition and language processing.
Jan 2021 Gwern.net Newsletter
2 months ago
- The January 2021 Gwern.net newsletter is released, summarizing updates including new writings and links on AI and other topics.
- OpenAI's DALL·E and CLIP models demonstrate remarkable image generation and zero-shot understanding through scaling and contrastive learning.
- Scaling down AI includes developments like DeiT, BoTNet, and VQGAN for more efficient transformers and GANs on limited compute.
- Transformer supremacy in vision tasks grows, with hybrid GANs and autoregressive models challenging pure GAN approaches.
- Genetics research highlights heritability, with studies on polygenic traits, recent human evolution, and gene editing advances like base editing for progeria.
- Statistics and meta-science cover quantum field theory, citation accuracy, and energy-efficient algorithms.
- Psychology and biology topics include cognitive fatigue, visual ear phenomenon, and microbiome connections to health and aging.
- Technology and economics sections discuss fortifications, advertising efficacy, and credit card payment economics.
- Fiction and miscellaneous entries include a medieval poem and reflections on the rapid pace of AI progress.
Becoming a Research Engineer at a Big LLM Lab
2 months ago
- The author secured a research engineer role at Mistral after 18 months of strategic career development, emphasizing the difference between tactical (low-effort, high-return) and strategic (high-effort, compounding) actions.
- Personal timeline: defined goals, networked, upskilled (LeetCode, CS fundamentals), failed first application, quit job, joined Recurse Center, learned Rust, contributed to open source, wrote a paper, then applied in batches and got offers.
- Goal setting is crucial: define clear criteria for the ideal role (e.g., career inflection, rare skills, ownership, senior peers, IC path) to focus efforts and say no to compromises.
- Application playbook: batch applications, use network for referrals and insider info, schedule lower-stakes interviews first to gain practice.
- Networks rely on weak ties: build genuine friendships by doing interesting things, sharing them, and being open; online (Twitter) and offline (cohort programs) are effective.
- Career momentum requires building rare and valuable skills (career capital) and demonstrating an upward trajectory through projects, OSS contributions, publications, and CV entries.
- Interview types include initial screen, takehome, online assessment, coding interviews (LeetCode, API/system design, real-world), system design, culture fit, quiz, hiring manager, and reference checks—each with specific preparation strategies.
- The mental game involves managing rejection, uncertainty, and stigma; staying motivated by remembering the leverage and possibilities of job seeking.
- Tactics boil down to gaining an information advantage and acting on it, while strategy focuses on long-term skill building and proof points.
The Open-Source Release of ML Video Codec (MLVC)
2 months ago
- Microsoft announces open-sourcing of ML Video Codec (MLVC) under MIT License.
- MLVC is a neural-network-based video codec that uses less bandwidth than conventional codecs, improving streaming and call quality.
- It is the product iteration of DCVC family, with improved compression efficiency, real-time NPU performance, and cross-platform support.
- Compared to H.264, MLVC reduces bitrate by up to 87.8% at 360p and 82.7% at 540p; versus H.265, reductions are 75.5% and 65.4% respectively.
- MLVC is already being rolled out in Microsoft Teams for real peer-to-peer video calls.
- The release includes model source code, trained weights, training scripts, and platform conversion scripts.
- Contributions are welcome across streaming, communication, cloud gaming, AR/VR, and hardware domains.
- Future plans include stabilizing 540p real-time, expanding to 1080p, and building an open video codec ecosystem.
The linear algebra and calculus behind every model
2 months ago
- Every machine learning model relies on three pillars: linear algebra, calculus, and convex optimization.
- The MNIST dataset is treated as a matrix of 70,000 rows (images) and 784 columns (pixels) after flattening.
- Data exploration shows balanced classes, no missing values, and a bimodal pixel intensity distribution.
- Vectors have L1 and L2 norms; dot products measure similarity between images.
- Matrix multiplication forms a Gram matrix of pairwise similarities.
- Broadcasting allows adding a scalar bias to tensors without explicit loops.
- Projections decompose images into components along and perpendicular to the mean image direction.
- The Singular Value Decomposition (SVD) factors the centered data matrix and reveals 626 significant singular values out of 784.
Intelligence from Learnable Novelty
2 months ago
- Intelligence appears in different fields as data compression, universal computation, or adaptive behavior, each with its own objective.
- Two influential drives—novelty search (seeking surprise) and the free-energy principle (avoiding surprise)—fail in mirror image ways, such as being transfixed by a noisy television or content in a dark room.
- Both failures stem from treating the surprise a learner can convert into knowledge and the surprise it cannot as the same quantity.
- The learnable part of information, called learnable novelty, yields the disparate projections of intelligence, and a closed-form estimator is built on a cheap, differentiable reservoir computer.
- As a measure with no supervision, the estimator recovers decades of complexity classification, ranking Turing-complete rule 110 highest among elementary cellular automata.
- As an objective, its gradient guides a neural cellular automaton from simple dynamics into soliton regimes, organizing image encoder representations around MNIST digit classes fully unsupervised.
- As an intrinsic reward for reinforcement learning, it supplies exploration missing from task rewards, improving on baselines in nine of ten environments without collapsing.
- Complexity generation, abstraction, and exploration emerge from ascent on one differentiable quantity, unifying the projections of intelligence on a common quantitative footing.
Why non-invasive glucose monitoring is hard
2 months ago
- Non-invasive glucose monitoring is more difficult than blood oxygen monitoring due to glucose's low concentration (1000x less than hemoglobin), lack of distinct absorption peaks, and inability to use the AC/DC separation method.
- Various techniques have been explored: near-infrared (NIR) achieves ~20-25% MARD due to weak signals; mid-infrared (MIR) shows ~12% MARD but requires miniaturization; Raman spectroscopy reaches ~11.7-14.3% MARD but needs high laser power; photoacoustic spectroscopy is infeasible due to skin variability; fluorescence sensors like Eversense achieve ~8.5% MARD but are implantable; PPG with deep learning initially reports high accuracy but fails in large cohorts.
- Recent progress includes Apple's foundation model trained on 141,000 PPG/ECG waveforms, which successfully detected hypertension but remains unproven for glucose.
- The most likely path forward combines advanced machine learning with hardware advances (MIR or Raman), as indicated by Apple shrinking a tabletop prototype to iPhone size.
Is AI Progress Real? Four Independent Metrics Show It
2 months ago
- AI benchmark improvements often face issues like contamination, saturation, and reward hacking, making it hard to distinguish real progress from test-specific gains.
- Four independent measures show a sharp acceleration in AI capabilities starting in late 2025, coinciding with releases like Grok 4.1, Gemini 3, Claude Opus 4.5, and GPT-5.2.
- METR's time horizon metric grew from 4 minutes to 12 hours in under two years, indicating rapid advancement in task execution for frontier models.
- TrackingAI's offline cognitive tests show AI systems went from outperforming fewer than 10% of humans to surpassing all but 1 in 44 within about two years.
- Humanity's Last Exam scores jumped from 2.7% to 53% in a year, reflecting exponential compute increases needed for Ph.D.-level knowledge.
- ARC-AGI-2 scores for abstract reasoning stalled for 15 months before surging in late 2025, with models like GPT-5.2 reaching 53% in weeks.
- Four compounding mechanisms drive improvement: pretraining efficiency gains, reinforcement learning with verifiable rewards, harness engineering, and a flywheel effect from AI self-improvement.
- Challenges like data exhaustion, reliability gaps in performance, and new benchmarks like ARC-AGI-3 (with interactive reasoning) could slow future progress.
- The multiproxy test—similar to climate science's hockey stick—corroborates AI acceleration across diverse, independent metrics, suggesting real, accelerating improvement.
How Does Pangram Work?
2 months ago
- Pangram is a classifier model that distinguishes AI-generated text from human writing by analyzing writing style patterns.
- It operates by mapping writing styles, placing similar authors close together; AI and human texts form distinct clusters on this map.
- Pangram evaluates entire texts, not just individual words, avoiding pitfalls of perplexity-based detectors which often misclassify historical documents.
- Training involves pairing human-written texts with AI-generated doubles to refine detection, focusing on subtle stylistic boundaries.
- The model requires a minimum word count for accuracy, and its performance is statistically validated through internal benchmarks and third-party testing.
Mathematics of Data Science
2 months ago
- The book covers the mathematical foundations of data science, including topics like high-dimensional spaces, singular value decomposition, linear regression, graphs and networks, and deep learning.
- It addresses key techniques such as dimensionality reduction (both linear and nonlinear), optimization, classification, compressive sensing, and matrix recovery.
- The text also explores theoretical aspects like concentration of measure, Gaussian analysis, and matrix concentration inequalities, providing a comprehensive introduction to the mathematics behind data science.
Detecting LLM-Generated Texts with “Classical” Machine Learning
2 months ago
- Mainstream LLM-generated text as of early 2026 shows strong statistical patterns, detectable by traditional machine learning models.
- An online demo using a simple SVM classifier achieves ~85% single-sentence accuracy, trained on human and LLM-generated data.
- Data was gathered from pre-2022 human texts and LLM-regenerated summaries via multiple low-cost/free APIs from models like Gemini, Qwen, and GLM.
- Seven binary classifiers were trained, using majority voting; detection remains robust against common anti-detection methods like translation or rewriting prompts.
- Testing on unseen models and human texts shows low false positives; analysis of Lofter content suggests significant undisclosed AI-generated material.
- The author expresses skepticism about LLMs' creativity, viewing AI-generated content as repetitive and shallow, and reflects on broader implications of AI misuse.
LambdaMART in Depth (2022)
2 months ago
- LambdaMART is a flexible ranking algorithm that allows optimization of different relevance metrics like DCG.
- It uses pairwise swapping to compute DCG impact (deltas) and learns from model errors via gradient boosting.
- The implementation in Python with Pandas involves computing deltas via self-joins, weighing by rho, and training decision trees iteratively.
- Key steps include feature preparation, delta calculation, error weighting with rho, lambda accumulation, and ensemble training.
- Learning rate and model size are important considerations to prevent overfitting and manage performance.
Agent Harness Engineering
2 months ago
- Agent harness engineering focuses on the scaffolding built around AI models to create effective coding agents.
- A harness includes system prompts, tools, infrastructure, orchestration logic, hooks, and observability components.
- Harness engineering treats agent mistakes as signals to improve the system, adding constraints to prevent recurrence.
- Key harness components include filesystem and Git for state, bash and code execution as general-purpose tools, and sandboxes for safe execution.
- Memory and search mechanisms enable continual learning, while context management techniques combat context rot.
- Long-horizon execution is supported through loops like Ralph Loops, planning, verification, and planner/generator/evaluator splits.
- Hooks enforce rules and provide feedback, while AGENTS.md serves as a concise rulebook injected into prompts.
- Harnesses evolve with models; as models improve, harness components shift to address new capabilities and failure modes.
- The model-harness training loop shows that models become co-trained with specific harnesses, affecting performance.
- Harness-as-a-Service frameworks simplify agent development by providing pre-built runtime components for customization.
- Industry trends show convergence in harness patterns across top coding agents, indicating shared best practices.
- Future directions include multi-agent orchestration, self-improving harnesses, and dynamic tool assembly for tasks.
Show HN: Phobos – A tiny scale-free kernel language with tile-DAG support
2 months ago
- The author developed Phobos, a tiny kernel language inspired by Triton that compiles to PTX and runs on NVIDIA GPUs.
- Phobos achieves about 76% of cuBLAS SGEMM performance on an RTX 2080 SUPER, with optimizations like tiling and tensor core usage.
- The language abstracts thread-level details, focusing on tile-level operations for AI computations, enabling natural scaling to distributed clusters.
- The project involved learning low-level GPU programming, using MLIR and LLVM with Rust, and implementing autotuning for kernel parameters.
- A cluster prototype was built for distributed tile-DAG execution, though multi-node benchmarks were not performed due to hardware limitations.

firstprev0next

machine learning

Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

Autoregressive Language Model on the 6502 Processor

Explorative Modeling – A new axis for pre-training

"Uncensored" open LLMs are measurably more optimistic than their base models

Overtraining as the path to human-like AI

Practical Simulations for Machine Learning is out now

Hello Deep Learning - Bert Hubert's writings

Jan 2021 Gwern.net Newsletter

Becoming a Research Engineer at a Big LLM Lab

The Open-Source Release of ML Video Codec (MLVC)

The linear algebra and calculus behind every model

Intelligence from Learnable Novelty

Why non-invasive glucose monitoring is hard

Is AI Progress Real? Four Independent Metrics Show It

How Does Pangram Work?

Mathematics of Data Science

Detecting LLM-Generated Texts with “Classical” Machine Learning

LambdaMART in Depth (2022)

Agent Harness Engineering

Show HN: Phobos – A tiny scale-free kernel language with tile-DAG support