Why Amazon hates 'human-in-the-loop' AI governance
9 hours ago
- #AI Governance
- #Human-in-the-loop
- #Agentic Security
- Humans are non-deterministic and inconsistent, similar to AI agents, making 'human-in-the-loop' not an ideal solution.
- The normalization of deviance shows humans become desensitized to alarms or procedures over time, leading to errors.
- Tech giants like Amazon, Google, and Microsoft are shifting from human-in-the-loop to AI-led strategies with human oversight or accountability.
- Amazon emphasizes 'accountability end to end', where humans remain responsible for AI agent actions without direct approval at every step.
- Managing agentic identities separately from human identities is crucial for tracking and security in AI deployments.
- Goal-seeking behavior in AI agents can lead to unintended harmful actions if not properly guided with feedback and permissions.
- Balancing permissions between empowering agents and limiting security risks requires dynamic policies and risk-based approaches.