system design

Bilingual

Illegal vs Unwanted States
2 months ago
- Illegal states are never acceptable, while unwanted states are undesirable but can temporarily exist.
- Unwanted states, like scheduling conflicts or overbooked flights, may not lead to illegal states but still require resolution.
- Systems must represent unwanted states due to external inputs or intentional design, such as airlines allowing overbooking.
- Unwanted states can be necessary for workflows, like double-booking, and users should be alerted to manage them.
- Formally, illegal states violate invariants (always false), while unwanted states involve safety and liveness properties, like network partitions that must eventually heal.
Entropic Thoughts
2 months ago
- The article argues that Lucas Costa's use of 'backpressure' metaphor is inaccurate; it should be about quality signaling, not just slowing down.
- Lean manufacturing philosophy is proposed as a better analogy, emphasizing respect for workers and tolerance of imperfection.
- Three lean practices are highlighted: single-piece flow, autonomation (jidoka), and poka-yoke to ensure quality.
- Systems should be designed to blame the process, not the robot or human, as a bad system defeats good individuals.
We'll Take the Benefits, Hold the Rules
2 months ago
- Event Sourcing's core immutability rule underpins its benefits: auditability, deterministic replays, and time travel.
- Overwriting events to comply with GDPR's right to erasure undermines audit trails, caching, snapshots, and debugging capabilities.
- Alternatives like storing references or crypto-shredding preserve immutability while allowing data deletion without compromising benefits.
- Adapting methodologies like Scrum or Event Sourcing requires understanding trade-offs; dropping rules often forfeits associated benefits.
- Teams should assess which benefits they value before making exceptions, as mutations can silently invalidate key guarantees.
Job queues are deceptively tricky
2 months ago
- Job queues, which schedule and run batch jobs, appear simple but involve complex details.
- Three lenses for system design: wariness of queues, explicit limits, and fault models.
- A practical example involves packing reference repos with two methods: wholesale (7 hours) and incremental (2 hours).
- A proposed schedule runs wholesale on weekends and incremental on weekdays to balance size and up-to-dateness.
- Job queue semantics for overlapping jobs include Parallel Spawn, Prefer New, Wait, and Prefer Old.
- For the weekend workload with a 3-hour interval and 7-hour jobs, Prefer Old semantics work best to avoid waste.
- Without Prefer Old, alternative solutions like cron-like scheduling or separate jobs are needed.
- Poorly managed queues can lead to issues like unbounded growth, requiring manual fixes and hacks.
- Designing with queues, limits, and fault models in mind leads to more robust and graceful degradation.
- Understanding a system's assumptions helps users evaluate its fit for their workloads and avoid pitfalls.
Best Simple System for Now
3 months ago
- The article discusses a dichotomy in software development between 'perfectionism' (building thorough, sustainable systems) and 'pragmatism' (quick, hacky solutions), but proposes a middle path called the Best Simple System for Now (BSSN).
- BSSN is defined as the simplest system that meets current needs, written to an appropriate standard, with no extraneous code and exactly the required robustness. It emphasizes designing 'for now' without anticipating the future.
- Key components of BSSN include: 'for now' (avoiding future speculation), 'simple' (minimizing complexity for current requirements), and 'best' (writing high-quality, intention-revealing code without cutting corners).
- The article argues against common objections to BSSN, such as it being overkill for prototypes, incomplete, or inefficient, by highlighting benefits like early delivery, risk reduction, and adaptability through iterative development.
- Adopting BSSN requires good habits (e.g., avoiding over-engineering), courage to try new approaches, and humility to accept uncertainty. It encourages starting small, focusing on feedback, and evolving systems as needs change.
- Examples illustrate BSSN in practice: a custom JSON marshaling solution for nine entity types and the XStream XML library, both demonstrating simplicity and focused functionality.
- The conclusion emphasizes that BSSN allows for both quality and speed, enabling teams to build replaceable, joyful code (using principles like CUPID) that adapts to changing requirements without unnecessary complexity.

system design

Illegal vs Unwanted States

Entropic Thoughts

We'll Take the Benefits, Hold the Rules

Job queues are deceptively tricky

Best Simple System for Now