Using Claude Code SDK to reduce E2E test time
4 days ago
- #Claude Code SDK
- #CI/CD Optimization
- #E2E Testing
- E2E tests are slow but essential for verifying complete user workflows.
- Running only relevant E2E tests for specific code changes can reduce test time significantly.
- Glob patterns can be used to match file paths and trigger relevant tests, but they require maintenance and can be imprecise.
- Claude Code SDK uses tool calls to strategically examine files, search for patterns, and trace dependencies to determine relevant E2E tests.
- The solution involves analyzing PR modifications, maintaining an E2E test inventory, and crafting a precise prompt for Claude Code.
- The final pipeline combines git diff, E2E test files, and a prompt to generate test recommendations.
- Results show a reduction in E2E test time from 44 minutes to under 7 minutes for most PRs.
- Claude Code effectively catches all relevant E2E tests, though it may run more tests than necessary.
- The solution costs about $30 per contributor per month but saves money on device farm runners and prevents production bugs.