- The article introduces Antidoom, a method using Final Token Preference Optimization (FTPO) to reduce doom loops—repetitive spans during inference—in language models.
- Doom loops arise from overtrained tokens, prior context reinforcement, and greedy sampling, especially in small reasoning models on hard problems.
- Antidoom targets the first token of a loop, training with chosen/rejected token pairs to prefer coherent alternatives while minimizing distributional disruption.
- Results show doom-loop rates dropping significantly (e.g., from 10.2% to 1.4% in LFM2.5-2.6B), improving eval scores without teaching new content.
- Multiple rounds of Antidoom may be needed, as fixing loops can expose new failure points, and optimal performance often occurs with near-greedy sampling after training.
- The method is implemented via a pipeline with code available on GitHub, requiring careful hyperparameter tuning and early stopping to avoid over-training.