Writing a storage engine for Postgres: an in-memory Table Access Method
20 days ago
- #Database Development
- #Storage Engines
- #Postgres
- Postgres 12 introduced the ability to swap out storage engines, similar to MySQL's long-standing feature.
- Different workloads benefit from different storage approaches, such as columnar storage for analytics and LSM trees for write-heavy workloads.
- Table access methods (TAMs) offer lower-level integration than foreign data wrappers (FDWs), potentially providing better performance and cleaner integration.
- The article provides a step-by-step guide on implementing a minimal in-memory storage engine for Postgres, including setting up a debug build and writing the necessary C code.
- Key methods to implement include `slot_callbacks`, `beginscan`, `getnextslot`, and `tuple_insert` for basic table operations.
- The example demonstrates how to store and retrieve data from an in-memory table, handling both inserts and selects.
- Future directions include exploring more complex storage engines like CSV or RocksDB, and understanding indexes, deletions, and updates.