Don't rewrite your CLI for agents
4 hours ago
- #agent performance
- #JSON payload
- #CLI arguments
- Advice to replace CLI args with a single JSON payload for agents may not hold up under measurement.
- Testing a synthetic CLI showed that flat CLI arguments achieved perfect correctness across all model runs, while JSON had lower correctness for smaller models.
- JSON mode costs 4x to 11x more per task due to extra tokens from retries and output, with shell escaping being a major failure mode.
- CLI arguments constrain the input space, helping models produce correct output consistently, especially smaller ones.
- The recommendation is to keep CLI arguments for agent use, as they are more reliable, cheaper, and avoid shell-dependent issues.