Distilling proprietary model reasoning into open-source search agents
a day ago
- Agentic search enables LLMs to solve knowledge-intensive tasks through multi-step reasoning and retrieval, but outcome-based RL provides sparse supervision.
- Knowledge distillation from proprietary models can densify supervision, but logit-matching is hindered by hidden logits and mismatched tokenizers, and trajectory imitation transfers stylistic artifacts rather than reasoning competence.
- MAPD (Multi-Agent Protocol Distillation) is a joint distillation and RL framework that uses a structured, style-normalized JSON protocol as an intermediate representation.
- An offline multi-agent system decomposes queries, retrieves evidence, repairs failed searches, and converts traces into protocols containing task type, reasoning plan, and grounding facts.
- During training, the protocol is provided to a privileged branch of the student policy, yielding dense distillation signal alongside sparse RL objective.
- MAPD outperforms competitive baselines on 7 QA benchmarks, achieving 39.4% success rate on Qwen3-1.7B and 44.4% on Qwen3-4B, and generalizes robustly across proprietary teachers while mitigating style drift and verbosity degeneration.