Autoregressive or Diffusion Language Models, Why Choose?
8 days ago
- #diffusion-models
- #language-models
- #autoregressive-models
- TiDAR is a hybrid architecture combining diffusion and autoregressive models for language generation.
- It drafts tokens in parallel using diffusion and samples final outputs autoregressively in a single forward pass.
- TiDAR achieves higher throughput (4.71x to 5.91x more tokens per second) compared to autoregressive models.
- It outperforms speculative decoding and diffusion variants in both efficiency and quality.
- TiDAR is the first architecture to close the quality gap with autoregressive models while maintaining high throughput.
- The model is designed for serving-friendly deployment with low overhead.