Enabling small language models to solve complex reasoning tasks
3 days ago
- #MIT Research
- #AI
- #Language Models
- Language models (LMs) struggle with complex tasks like Sudoku, advanced puzzles, and math proofs despite excelling in simpler tasks.
- MIT researchers developed DisCIPL, a collaborative framework where a large LM plans and delegates tasks to smaller LMs for efficiency and accuracy.
- DisCIPL uses LLaMPPL, a programming language, to communicate instructions and constraints to smaller models, improving their responses.
- The system outperforms leading models like GPT-4o and o1 in accuracy and efficiency, with significant cost savings.
- DisCIPL excels in tasks requiring strict rule-following, such as writing constrained sentences, creating grocery lists, and planning travel itineraries.
- Smaller LMs in DisCIPL are cheaper and scalable, allowing parallel execution for a fraction of the cost of larger models.
- Future plans include expanding DisCIPL for mathematical reasoning and fuzzy preference tasks, and testing with larger models.