I burned all my tokens researching how to save tokens
15 hours ago
- The author initially used Claude Fable 5 for deep research on tokenomics, but it burned through the token limit in 30 minutes without producing results, prompting a need for optimization.
- To reduce costs and improve efficiency, the author leveraged existing subscriptions (Claude, Codex, Antigravity) with shared memory via a custom setup, allowing multiple models to work together without extra payment.
- A model-orchestration pattern was implemented, assigning specific roles to cheaper models (e.g., Claude Sonnet 5 for finding, Claude Opus 4.8 for verification) and using Claude Fable 5 only for planning, which extended research time from 30 minutes to several hours.
- Hallucinations were reduced by enforcing rules such as separating finding and verification agents, requiring URLs and quotes for claims, and avoiding unsourced numbers, with rules evolving based on mistakes caught during research.
- The /deep-research tool was moved to the end of the pipeline, working on pre-verified findings to use fewer tokens and improve efficiency, reducing from 111 agents in 30 minutes to 61 agents in 22 minutes.
- A trusted knowledge base was built in Obsidian using an LLM wiki pattern, with automated checks and human oversight, though initial rules sometimes incorrectly rejected valid projects, highlighting the need for hybrid human-AI collaboration.
- Key findings from the research include: harness design can significantly impact token usage (e.g., ~66x spread), context compaction may increase costs by triggering loops, mid-session tool changes can silently rebill cached content, and invoice costs often exceed simple token estimates due to hidden factors like framework overhead.
- The author recommends using cheap models for initial research, accurate models for verification, and deep research last, while leveraging multiple subscriptions with defined roles to avoid over-reliance on a single expensive model.