System tests
a day ago
- The author argues that system tests are essential for ensuring the entire system works together, despite debates over terminology.
- Keygen's test suite takes over 2 hours on a single core, with most time spent on authorization and system tests.
- System tests are criticized for being slow and brittle, but the author believes they catch bugs that unit tests miss, especially in business logic within middleware and extensions.
- The author uses Gherkin and Cucumber for system tests, finding them effective for JSON APIs, though acknowledges they may be less suitable for HTML-rendered apps.
- System tests can be parallelized to reduce time; Keygen's tests run in ~10 minutes on 32 cores vs. 2 hours on one core.
- Without system tests, there is no guarantee that all components are correctly wired together, leaving potential bugs to be caught by customers.