I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
4 hours ago
- #Performance Optimization
- #SQL Parser
- #AI Development
- PostHog's author rewrote the SQL parser using AI agents, resulting in a 70x speed increase.
- The new parser is a hand-rolled recursive-descent with Pratt expression core, replacing ANTLR-generated code.
- Property-based testing and AI-driven iteration were key to matching the oracle parser's output.
- The parser handles untrusted input and is critical for SQL transpilation to ClickHouse.
- Future parser development may shift from generators to AI-assisted hand-rolling with fuzzing.