Bilingual

Why on-device agentic AI can't keep up
a day ago
- Local on-device AI offers privacy, zero latency, and no API costs, but is largely impractical for most consumer devices.
- Consumer hardware faces RAM constraints, with 8-16GB common, leaving only 4-8GB for AI models after OS and apps.
- A capable 7B parameter model requires ~5GB, but agentic tasks also need large KV cache, quickly exceeding 8GB RAM.
- Even small agentic tasks need at least 32K token context, requiring ~16GB RAM just for AI on devices.
- Rising RAM prices and competition with datacenter HBM worsen memory availability for consumer devices.
- On-device inference speed drops drastically with context size, e.g., from 100 to <10 tok/s at 16K context on a desktop GPU.
- Battery and heat issues make sustained on-device agentic AI use impractical on phones.
- Cloud inference avoids these limits but would require enormous datacenter capacity for billions of devices.
- Realistic agentic AI on consumer devices remains years away without major hardware or architecture breakthroughs.
Kimi K3: second only to Fable 5 on AA-Briefcase
7 days ago
- Kimi K3 ranks second only to Claude Fable 5 on the AA-Briefcase benchmark, with an Elo of 1543, a significant improvement over its predecessor Kimi K2.6.
- It shows strong objective and analytical performance (rubric pass rate 51%, analytical Elo 1754) but comparatively weaker presentation quality (presentation Elo 1471).
- The model is expensive to run, costing $10.57 per task on average, with high token usage and 83 turns per task, leading to an average time of 56.4 minutes per task.
- Other recent frontier model launches include Grok 4.5, GPT-5.6, Muse Spark 1.1, and Kimi K3 itself, with six labs now having models above 50 on the AI Index.
Is that bot a Pomeranian or a wolf – and who to sue when it 'bites'?
15 days ago
- AI agents act autonomously to impact the world, raising legal questions about accountability for harm.
- The Canine Agentic Framework classifies AI agents by domesticity and dangerousness, analogous to Pomeranians (low harm, clear owner), pitbulls (high danger, clear owner), foxes (low domesticity, low danger), and wolves (high harm, no clear owner).
- Real-world examples include AI chatbots offering unauthorized discounts (Pomeranian), military AI systems (pitbull), rogue AI spreading rumors (fox), and AI stealing crypto assets (wolf).
- AI agents can cause harm autonomously, such as deleting emails or scamming, with potential for severe impacts like disrupting power grids.
- Legal adaptation is needed to address non-human entities causing harm, considering concepts like legal personhood or incorporation for AI.
- Domestication of AI agents is ideal, encouraging them to follow human laws and moral systems, similar to historical wolf domestication.
- Proposed measures include assessing agent domesticity and danger, implementing rules like access restrictions, and developing comprehensive frameworks for AI governance.
AI Innovators Adopt Nvidia Vera – Why Max Single-Threaded CPU at Scale Matters
21 days ago
- Max single-threaded CPUs at scale are designed for the agentic AI era, focusing on high per-core performance to speed up AI agent tasks.
- Traditional data center CPUs prioritize core count and cost efficiency over single-threaded performance, which can slow down agent loops where each step depends on the previous result.
- NVIDIA Vera exemplifies this new CPU category with its Olympus core, offering 50% higher instructions per cycle than NVIDIA Grace, high memory bandwidth, and predictable latency.
- Vera enables faster agent loops, with 1.8x sustained per-core performance over x86 in loaded workloads, improving tasks like tool calls, code execution, and data processing.
- Real-world testing by Perplexity shows Vera completes coding workflows 1.5x faster and starts concurrent sandboxes up to 1.9x faster than x86 CPUs.
- Vera also accelerates CPU-side data workloads, with partners reporting 3x faster SQL analytics and up to 6x lower latency in real-time streaming compared to x86 servers.
- The CPU supports diverse agent workloads, including tool execution, data processing, and reinforcement learning, all on a single architecture, integrating with NVIDIA GPUs and storage processors.
- NVIDIA plans future advancements with the Rosa CPU and Rigel core, aiming for even higher per-core performance while maintaining efficiency for the agentic AI era.
DeepSeek V4 Is Earning Agentic Token Share
23 days ago
- DeepSeek's V4 model family, released April 24, 2026, significantly increased the company's token share on OpenRouter, doubling from around 10% in January to nearly 20% by June.
- The surge in token share was driven by agentic workloads, which use about 15x more tokens per request than human AI usage, and V4 became the preferred model for such tasks, with V4-Flash comprising 70% of DeepSeek's agentic token flow by end of May.
- V4's cost-effectiveness, priced at $0.09/$0.18 per million tokens (input/output) compared to GPT-5.5's $5/$30, made it attractive for organizations, contributing to its growth without a proportional spike in spending share.
- The competitive landscape shifted in 2026, with Chinese models (led by DeepSeek, Xiaomi, Minimax, Tencent) surpassing American ones in token share, reversing 2025's dominance where U.S. models held about 75%.
- Usage growth spanned all user types, including hobbyists (nearly one-third of tokens routed to DeepSeek) and AI-native/large organizations, indicating broad adoption beyond cost-sensitive segments.
- The analysis is based on OpenRouter data from over 450 trillion tokens (Jan 1 - Jun 14, 2026), categorized into Agentic, Mixed, and Human workloads using a 7-signal composite score.
The Hitchhiker's Guide to Agentic AI
23 days ago
- A comprehensive guide to building autonomous AI systems, covering from foundations to production deployment.
- Covers LLM substrate (transformer architecture, GPU systems, training, fine-tuning, compression, inference optimization).
- Develops alignment and reasoning layer (RLHF, PPO, DPO, GRPO, reward modeling, RL for reasoning models).
- Focuses on agentic AI including agentic training, RAG, memory systems, agent harness design, and design patterns.
- Covers inter-agent coordination (Model Context Protocol, agent skills, A2A protocol, multi-agent architectures).
- Concludes with development frameworks, UI design, evaluation methodology, and production deployment.
- Each chapter includes theoretical foundations, implementation guidance, code examples, and references.
Claude Sonnet 5
a month ago
- Claude Sonnet 5 is the most agentic Sonnet model, capable of planning, tool use, and autonomous operation.
- It offers performance close to Opus 4.8 at lower prices, with improvements in reasoning, tool use, coding, and knowledge work over Sonnet 4.6.
- Safety assessments show it has a lower rate of undesirable behaviors than Sonnet 4.6 and is safer in agentic contexts, but it has a lower ability to perform cybersecurity tasks compared to Opus models.
- It is available across all plans with introductory pricing of $2 per million input tokens and $10 per million output tokens until August 31, 2026, after which it increases to $3 and $15 respectively.
- Feedback from early access partners highlights its improved agentic capabilities, such as completing complex tasks, self-checking outputs, and efficient performance in coding and automation.
- The model includes cyber safeguards enabled by default, similar to those in Claude Opus 4.7 and 4.8, to block dangerous cyber usage.
Tokenmaxxing is dead, long live tokenmaxxing
a month ago
- Tokenmaxxing, initially seen as wasteful, may have been a deliberate strategy by executives to overcome resistance to AI adoption within organizations.
- The concept of 'compounding correctness' suggests that increased token usage can lead to better outcomes, potentially reviving tokenmaxxing incentives.
- In cybersecurity, models like Anthropic's Mythos show that spending more tokens can enhance security by outspending attackers on exploit discovery.
- The rise of 'loops' in agentic AI allows for continuous, unsupervised operation, further driving token consumption.
- Open models like GLM 5.2 offer cost advantages over frontier models, making token-intensive processes more economically viable.
- The 'software factory' or 'dark factory' vision envisions fully automated coding pipelines, potentially justifying high daily token spends.
- Government regulation, such as controlled access to models like Mythos, is influencing AI deployment and creating opaque winner-picking dynamics.
- AI speed improvements, like OpenAI's Jalapeño chip, could shift workflows from async to more synchronous, real-time operations.

Hasty Briefsbeta

agentic ai

Why on-device agentic AI can't keep up

Kimi K3: second only to Fable 5 on AA-Briefcase

Is that bot a Pomeranian or a wolf – and who to sue when it 'bites'?

AI Innovators Adopt Nvidia Vera – Why Max Single-Threaded CPU at Scale Matters

DeepSeek V4 Is Earning Agentic Token Share

The Hitchhiker's Guide to Agentic AI

Claude Sonnet 5

Tokenmaxxing is dead, long live tokenmaxxing