Agents are not thinking: Science of agent behavior
2 days ago
- #search-space
- #agent-behavior
- #empirical-measurement
- Agents operate based on learned policies navigating a search space shaped by the Field, which dynamically changes with context.
- The Field is influenced by prompts, environment feedback, and system conditions, defining the boundaries within which agents search.
- Empirical measurement of agent behavior involves sampling trajectories to approximate the Field, as direct computation is intractable.
- A measurement function (φ) maps trajectories to fixed-dimensional vectors, capturing behavioral properties like tool calls, file reads, and edits.
- The choice of dimensions in φ is a hypothesis about which behaviors matter for understanding agent performance on a task.
- Metrics like width, convergence, separation, and skew help analyze the empirical Field, answering questions about behavioral variation and outcomes.
- Horizons filter trajectories by state (ψ), allowing analysis of behavior at different stages of task progression.
- Drift measures how failures diverge from successful trajectories, identifying problematic behavioral patterns.
- Intent (ρ_π) labels policy operational modes, revealing how agents alternate between acting and introspecting.
- Program strings compress behavioral fingerprints, grouping trajectories by structural similarity in act/think patterns.
- The framework enables empirical measurement of agent behavior, shifting focus from static instructions to dynamic search space shaping.
- Behavioral alignment metrics can be defined by prescribing desired behaviors and measuring adherence across models and prompts.