Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
9 days ago
- #Text Generation
- #PyTorch
- #Diffusion Models
- PyTorch implementation of FS-DFM for efficient text generation and discrete sequence modeling.
- Includes custom discrete solvers for flow matching and student-teacher distillation framework.
- Supports multiple solver options and various source distributions (uniform, mask).
- Requires Python 3.8+, CUDA 11.0+, and conda/mamba for setup.
- Training and evaluation scripts provided with configurable parameters.
- Transformer-based architectures with configurable vocabulary size and dropout regularization.
- Includes pre-training utilities and data loading components.
- Method to convert probability distributions to flow generators with safety features.
- Research paper citation provided for FS-DFM framework.
- Based on Meta's Flow Matching with custom discrete solvers added.