An oral history of Bank Python (2021)
6 hours ago
- #proprietary-systems
- #finance-technology
- #python-fork
- Bank Python is a proprietary fork of the Python ecosystem used by many major investment banks, differing significantly from standard Python.
- Minerva is a fictional, amalgamated Bank Python system, built around a global database called Barbara for storing Python objects, including financial data and source code.
- Barbara is a key-value store with hierarchical keys, using pickle and zip, offering strong consistency within instances and overlay features for multiple rings.
- Dagger manages directed acyclic graphs of financial instruments, automatically repricing derivatives and positions when underlying values change, similar to Excel's dependency updates.
- Walpole is a bank-wide job runner that handles long-lived services, periodic jobs, and deployments, with source code stored in Barbara, simplifying deployment via config files.
- MnTable is a proprietary table library used extensively for medium-sized data (gigabytes), offering memory-dense storage and efficient operations, unlike pandas.
- Bank Python systems are holistic and data-first, with a single codebase, database, and job runner, reducing accidental complexity but creating a monoculture that diverges from open-source Python.
- Challenges include a steep learning curve, skill atrophy, and proprietary tools (e.g., custom IDE), making interaction with external systems difficult and fostering NIH syndrome.
- The system is influenced by financial sector practices like Excel, aiming to provide a middle ground between spreadsheets and complex enterprise technology, enabling financiers to contribute.
- Lessons from such systems encourage rethinking software practices, highlighting the value of studying existing, real-world implementations.