ai agent safety

An Empirical Study: AI Agent Rules Need Context and Layered Enforcement

2 months ago

AI agent rules often depend on context and layered OS enforcement to be effectively checked, as demonstrated by ActPlane's study of 2,116 statements.
The study found that 64% of statements are policies requiring enforcement, with 83% being system-observable but only 45% directly OS-enforceable.
Cross-event policies, which depend on temporal ordering or data lineage, account for 16% of policies and are prevalent in development processes.
Context dependence is a major challenge: 64.2% of system-observable policies require project context to resolve concrete rules.
ActPlane uses a DSL with components like source, target operation, effect, temporal gates, and reason strings to compile policies into enforceable rules.
The enforcement engine leverages eBPF for OS-level checks, with effects including block, kill, and notify, and features like information-flow labels.
Semantic feedback significantly improves compliance, with ActPlane achieving a 75.8% decision compliance rate and a 97.7% recovery rate when violations are detected.
Overhead is low, with end-to-end costs under 8.4% even at 100 rules, and microbenchmarks showing syscall latencies orders of magnitude smaller than LLM inference times.
ActPlane's approach prevents 74% of unsafe behaviors in safety benchmarks but has limitations, such as coverage gaps for semantic-only policies and service-side effects.
The study highlights the need to start from existing instruction files, resolve context, and compile OS-enforceable subsets with feedback for effective agent governance.

Hasty Briefsbeta

ai agent safety

A Fireside Chat with Cat and Thariq from the Claude Code team

An Empirical Study: AI Agent Rules Need Context and Layered Enforcement