Inner Loop Agents
a year ago
- #AI Agents
- #LLM
- #Tool Use
- Inner loop agents allow LLMs to execute tool calls directly without client intervention.
- Regular LLMs rely on clients to parse and execute tools, while inner loop agents handle this internally.
- The LLM emits text with tool calls and a special token (<|eot|>) to signal completion.
- Software like Ollama and vLLM parse LLM output and manage the loop until the <|eot|> token is encountered.
- Inner loop agents enable concurrent tool use during the LLM's thinking process, enhancing efficiency.
- Models like o3 and o4-mini are trained to be agentic, optimizing tool use through reinforcement learning.
- Emergent tool use, where LLMs effectively use new tools without specific training, is still theoretical.
- Current options for tool use include MCP descriptions or training models specifically for tool use.
- Google's Agent 2 Agent (A2A) protocol facilitates communication between different fine-tuned LLM agents.
- Training an LLM with tools doesn't require the tools to be executed on the same host as the LLM.