We built an MCP server for your SRE agent
11 hours ago
- ClickStack MCP server improves AI agent accuracy by 18% and reduces tool calls by 26% compared to raw SQL.
- HyperDX built hdx-evals, an open-source benchmarking framework with deterministic synthetic telemetry for reproducible incident evaluations.
- Five incident scenarios test skills like root cause analysis, latency investigation, noisy signal identification, health checks, and segmented regression.
- Grading combines programmatic checks (40%), an LLM judge (60%), and a tool-error penalty, with blinded answers to ensure fairness.
- ClickStack MCP outperformed ClickHouse SQL MCP across all scenarios, with deltas from +7pp to +20pp.
- Tool descriptions, response design, and speed are critical for agent performance, with unclear errors often causing tool abandonment.
- Future plans include CI integration, expanded scenario coverage, and community contributions to the open-source framework.