token efficiency

Bilingual

The Economic Benefit of Refactoring
2 months ago
- Refactoring an agent-written codebase can significantly reduce token consumption for future changes.
- An experiment was conducted on a 17,155-line data access layer file, reducing it to multiple smaller files.
- Input tokens for a representative change dropped by 83% after refactoring, from 159,564 to 27,360.
- The savings are due to the agent reading less code, not because total code decreased.
- The refactoring process required human guidance, as agents were not good at refactoring.
- The experiment took about 8 hours and showed that refactoring is valuable in both time and money.
- The author suggests further research into more complex changes and continuous refactoring.
Which programming languages are most token-efficient?
2 months ago
- LLMs face context length constraints, making token-efficient programming languages beneficial for AI agents.
- Dynamic languages (e.g., Clojure) are more token-efficient than static ones (e.g., C), with a 2.6x gap.
- Functional languages like Haskell and F# are nearly as token-efficient as dynamic ones due to advanced type inference.
- Array languages like J (ASCII-based) are highly token-efficient, while APL's exotic symbols are poorly tokenized.
- Language selection for AI agents may increasingly prioritize token efficiency, impacting development sessions.
Which web frameworks are most token-efficient for AI agents?
2 months ago
- Minimal web frameworks (e.g., Express, Flask, ASP.NET Minimal API) are significantly more token-efficient for AI agents than full-featured frameworks, with a 2.9x token gap.
- All 19 tested frameworks produced working blog apps without bugs on the initial build, highlighting rapid progress in agentic development.
- Full-featured frameworks like SvelteKit and Django were the most efficient among their category, while Phoenix was the most token-expensive.
- Adding features (e.g., categories) cost roughly the same token band (15-30k) across most frameworks, regardless of their initial build cost.
- Framework overhead primarily impacts initial builds; extending existing code has similar costs everywhere.
- ASP.NET Minimal API stood out for token efficiency, static typing, speed, and low memory use.
- Popular frameworks like Rails and Laravel caused confusion for agents due to missing system packages.
<antirez>
2 months ago
- The user proposes a tag-based EDIT tool for local LLM agents, using line-number and checksum tags instead of verbatim text to save tokens and increase reliability.
- Current CAS (Check and Set) edit methods require the old text verbatim, causing token waste and failure with hallucinations or formatting issues.
- The new approach returns lines prefixed with a 4-character tag (checksum) and allows edits by line:tag pairs, reducing token usage especially for deletions.
- The tool is effective with models like DeepSeek v4 Flash, and the line:tag format helps LLMs exploit line information for ranges and follow-up calls.
- An alternative is using a file-level CRC32 tag with only line numbers, which saves tokens but fails on any unrelated changes, making tradeoffs apparent.
- The author suggests practical testing with ds4-agent across sessions to decide which method is better, with a command-line switch as an initial step.
Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
3 months ago
- Claude Code uses significantly more tokens than OpenCode for the same tasks, with baseline overhead around 33,000 tokens vs 7,000 for a simple reply.
- Cache inefficiency is a major factor; Claude Code rewrites tens of thousands of prompt-cache tokens mid-session, leading to up to 54x more cache writes than OpenCode.
- Configuration elements like instruction files and MCP servers add substantial token overhead, with a real setup reaching 75,000–85,000 tokens before user input.
- Subagents dramatically increase token usage; a task costing 121,000 tokens directly rose to 513,000 tokens when fanned out to two subagents.
- On multi-step tasks, Claude Code's batching can reduce total tokens compared to OpenCode's serial approach, though baseline costs remain higher.
- The token gap varies by model; Claude Code's system prompt is smaller for newer models like Fable 5, narrowing the overhead ratio.
- Prompt caching reduces costs but doesn't eliminate them; cache writes, reads, and context-window consumption still incur expenses.
- Claude Code exhibits cache prefix instability, leading to more frequent and costly cache rewrites compared to OpenCode's stable prefixes.
- Measurement was conducted via a logging proxy capturing exact JSON payloads and API usage, with results logged in an audit trail for integrity.
- The findings highlight the importance of monitoring token usage in production, especially under regulations like the EU AI Act.
ThinkingCap-Qwen3.6-27B: Qwen3.6 capabilities with 50% fewer thinking tokens
3 months ago
- The ThinkingCap-Qwen3.6-27B model is a finetuned version of Qwen3.6-27B, designed to reduce thinking tokens by 50% on average while maintaining answer quality.
- It can be used with libraries like Transformers and vLLM, and platforms such as Google Colab, Kaggle, and Docker for local deployment.
- Out-of-domain evaluations show token reductions of up to 67.8% on benchmarks like GPQA-Diamond, with maintained accuracy in areas like knowledge, math, and multimodal tasks.
- In-domain evaluations on holdout datasets like GSM8K and ARC-Challenge demonstrate even higher token efficiency, with reductions up to 74.1% and improved accuracy.
- Safety benchmarks indicate preserved guardrails, with refusal rates similar to the base model and token reductions of around 20-24% on safety prompts.
- The model supports GGUF quantization for efficient local inference via llama.cpp and compatible runtimes, with quantized versions available in a sibling repository.
- Usage examples include image-text-to-text tasks, and the model requires specific sampling parameters (e.g., temperature=1.0) as recommended for optimal performance.
Price per 1M tokens is meaningless
3 months ago
- Comparing AI model costs by price per 1M tokens is meaningless due to differences in tokenizers and token efficiency.
- Each frontier lab uses its own tokenizer, causing the same text to be split into varying token counts, making direct price comparisons unreliable.
- Token efficiency—how much is achieved per token—varies widely, especially with 'thinking' tokens in chain-of-thought processes, significantly impacting overall costs.
- A benchmark table shows models like GPT-5.5 can have lower cost per task despite higher token prices, while cheaper per-token models like GLM-5.2 may be less token-efficient.
- DeepSeek V4 Pro stands out for extremely low cost per task despite lower intelligence scores, highlighting cost-efficiency disparities.

token efficiency

The Economic Benefit of Refactoring

Which programming languages are most token-efficient?

Which web frameworks are most token-efficient for AI agents?

<antirez>

Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k

ThinkingCap-Qwen3.6-27B: Qwen3.6 capabilities with 50% fewer thinking tokens

Price per 1M tokens is meaningless