Getting Claude to QA its own work
14 hours ago
- #AI QA Tools
- #CI/CD Integration
- #Automated Testing
- Skyvern automates repetitive browser tasks like pulling invoices and filling forms, and can now be used to automate QA for code changes.
- A new MCP server with 33 browser tools was integrated with Claude Code, enabling it to self-QA frontend changes by interacting with the UI, increasing one-shot PR success from ~30% to ~70% and halving QA loops.
- Two skills were introduced: /qa for local testing and /smoke-test for CI, which analyze git diffs, generate test cases, run browser tests, and output results in a PASS/FAIL table.
- In CI, /smoke-test automatically tests PRs by starting the app, running targeted browser-based smoke tests, and posting evidence back to the PR, focusing on narrow test scopes to avoid flaky tests.
- The approach helps catch UI regressions like non-functional buttons or layout issues, but challenges remain in maintaining tests and determining appropriate test coverage for mixed changes.