The DuckDB ADBC Extension
6 hours ago
- #DuckDB
- #Apache Arrow
- #Database Connectivity
- The ADBC extension connects DuckDB to various external databases (Snowflake, Databricks, BigQuery, PostgreSQL, MySQL, etc.) using ADBC drivers built on Apache Arrow, enabling fast data transfer and interoperability.
- DuckDB previously relied on vendor-specific extensions, which were limited and added complexity; ADBC provides a universal API supporting over 30 systems, eliminating the need for separate extensions.
- The extension allows querying ADBC databases via read_adbc function or persistent connections with ATTACH, supporting operations like SELECT, INSERT, COPY, and CTAS, with features like connection pooling and metadata caching.
- To use the extension, install an ADBC driver (e.g., via dbc tool), create a connection profile, and then install/load the ADBC extension in DuckDB to query remote databases as if they were local.
- Limitations include autocommit-only mode (no multi-statement transactions yet) and no predicate/projection pushdown for ATTACH (though pushdown is supported via read_adbc and planned for future updates).