cost efficiency

Bilingual

AI-assisted analytics now 10x cheaper
2 months ago
- Small AI models are now as smart as prior flagship models, enabling new strategies for agentic analytics that prioritize speed and cost over raw intelligence.
- OpenAI's GPT 5.6 Luna, with an 80% price cut, achieves 99.8% accuracy on SQL benchmarks at a 5x lower price than Gemini-3-Flash, and even 10x cheaper on semantic modeling benchmarks due to model caching improvements.
- Luna occupies a new section of the price/performance curve, offering speed and cost efficiency for tasks that don't require maximum intelligence, such as SQL generation.
- Practical steps include switching to Luna on max effort for low-intelligence tasks and asking more questions (Jevons Paradox) to leverage the technology's improved performance.
- Larger shifts involve adopting faster analytical databases to avoid latency bottlenecks, building detailed context layers for weaker models, and running frequent evals with natural language questions to catch regressions and adapt to business changes.
- Eval workflows should be lab-agnostic to avoid lock-in and allow quick switching to optimal models as prices and performance evolve.
- The text introduces Guides, MotherDuck's context layer for AI agents, to convert raw intelligence into accurate answers with lower token spend.
Everyone says Chinese AI is dirt cheap
2 months ago
- Kimi K3 is not the cheap option; it costs similarly to Western flagships and performs mid-table on cost per test.
- Chinese models dominate as the 'doer' (coding execution), being 19 times cheaper than Western equivalents for the same results.
- Three traps include hidden billing for reasoning tokens, models failing in real tooling despite leaderboard rankings, and unstable routing on cheap aggregators.
- The optimal setup uses a frontier planner (e.g., Grok-4.5 or GPT-5.6 luna) paired with a cheap Chinese doer (e.g., DeepSeek flash).
- Cost should be measured per verified test pass, not per token.
Performing Live Migrations of VMs
2 months ago
- Sailboxes offer a cost-effective pricing model, charging only for active resource usage per second, making long-horizon agent workflows scalable.
- Live migration technology enables Sailboxes to move VMs between hosts seamlessly, with minimal downtime, supporting ephemeral compute and load balancing.
- A network proxy handles open connections during migrations, spooling packets to maintain connectivity and providing additional features like whitelists and credential injection.
- P2P migrations use userfaultfd for efficient memory transfer, paging in data on demand and using larger memory chunks to reduce latency.
- Elastic memory consumption via virtio-mem allows Sailboxes to dynamically adjust memory usage, optimizing resource allocation and speeding up migrations.
- Sailboxes charge $0.015/vCPU-hour, significantly cheaper than competitors, with billing based solely on active usage for cost-effectiveness.
- Trade-offs include occasional latency spikes during migrations, but these are infrequent and manageable for long-horizon agents.
- Sailboxes are now generally available, with ongoing infrastructure improvements planned for fleet utilization and caching systems.
Distributed System Is Slower Than a Laptop
3 months ago
- A 2015 paper showed single-threaded laptop outperformed distributed systems on large-scale graph processing tasks.
- Many distributed systems fail to beat a single competent thread even with hundreds of cores, leading to high COST (Configuration that Outperforms a Single Thread).
- Companies pay millions for distributed systems when simpler, cheaper single-machine solutions suffice and perform better.
- Hardware improvements have made single servers extremely powerful, but distributed-by-default reflexes persist without justification.
- Distributed systems often introduce unnecessary complexity, higher costs (compute, engineering, incidents), and slower performance.
- Frameworks, career incentives, vendor pricing, and misconceptions about scaling drive unnecessary distribution.
- Legitimate reasons for distribution include data size, availability needs, geographic latency, and organizational partitioning.
- Building a single-machine baseline before adopting distributed architectures can save costs and improve design decisions.
A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
3 months ago
- Verification loops significantly improve coding agents' ability to complete multi-step web development tasks by catching bugs early.
- Using IronBee (a verification layer) with DeepSeek (low-cost model) increased task completion from an average of 6.8 to 17 out of 20 tasks, matching the performance of Claude Opus (frontier model).
- DeepSeek with IronBee achieved similar results to Opus alone but at about 1/7th the cost ($2.4 vs. $15 per run), showing verification can reduce reliance on expensive models.
- The study emphasizes that verification helps most with bugs that only appear when the running application is tested, especially in sequential tasks where early errors compound.
- Limitations include using only one project and model pair initially, with plans to expand testing to more models, projects, and a cost-matched ablation study for broader insights.
GLM5.2 on AMD MI355X at 2626 tok/s/node at over 2x lower cost than Blackwell
3 months ago
- AMD MI355X GPUs offer significantly lower cost per GPU (about 2.75x cheaper) compared to NVIDIA Blackwell B300, providing a cost-effective solution for AI inference.
- Wafer achieved 2626 tok/s/node aggregate throughput with GLM5.2 on AMD MI355X at 2.4 RPS, achieving 80% of B200 performance at over 2x lower cost, and 213 tok/s in single-stream decoding.
- Optimizations included quantizing GLM-5.2 to lossless MXFP4 with AMD Quark, using sglang as the inference framework, fixing speculative decode issues, and tuning MoE kernels to improve prefill performance.
- AMD faces challenges with day-0 support and software friction for frontier models, but the performance gap is narrowing as kernel and model optimizations improve, reducing reliance on custom kernels.
- The study highlights that achieving state-of-the-art performance on AMD is increasingly about support and optimization rather than inherent software limitations, indicating erosion of NVIDIA's CUDA advantage.
Nano Banana 2 Lite
3 months ago
- Nano Banana 2 Lite is a high-speed, cost-efficient Gemini Image model designed for fast image generation and editing.
- It enables rapid prototyping and iteration, allowing creators to generate thousands of images at a low cost compared to heavier models.
- The model maintains control and accuracy, supporting character consistency, precise visual editing, and real-world knowledge integration.
- Example applications include Space Lift for interior design, Gridscape for interactive learning, Peek-A-Word for educational visuals, and Anywhere for travel postcards.
- Performance benchmarks show strong results in image editing and generation, with low latency per 1k resolution image.
- Testimonials from companies like Figma Weave, Manus AI, Artlist, Weekend, and Latitude highlight its speed, reliability, and quality for creative and real-time workflows.
- Users are advised to provide detailed prompts for better results and to verify outputs, as the model may struggle with fine details, text accuracy, and complex edits.
- Safety features include content filtering and SynthID for watermarking AI-generated images, with ongoing improvements in safety and consistency.
- Legal disclaimers note that LLMs like Gemini 3.1 Flash-Lite Image may produce inaccurate or offensive content and should not be used for professional advice.

cost efficiency

AI-assisted analytics now 10x cheaper

Everyone says Chinese AI is dirt cheap

Performing Live Migrations of VMs

Distributed System Is Slower Than a Laptop

A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost

GLM5.2 on AMD MI355X at 2626 tok/s/node at over 2x lower cost than Blackwell

Nano Banana 2 Lite