Pg_parquet v0.4.0: Google Cloud Storage, HTTPS storage, and more
a year ago
- #Data Integration
- #PostgreSQL
- #Parquet
- pg_parquet v0.4.0 released with new features including Google Cloud Storage and HTTPS support.
- New capabilities: COPY TO/FROM Google Cloud Storage, http(s) stores, stdin/stdout with FORMAT PARQUET.
- Supports Parquet UUID, JSON, and JSONB types for enhanced data handling.
- pg_parquet simplifies exporting and importing Parquet files within Postgres without third-party tools.
- Parquet is an open standard file format with columnar compression, ideal for data movement.
- Apache Iceberg recommended for full database cloud storage with metadata layer.
- Example SQL commands provided for copying data to/from Parquet files and cloud storage.
- Version 0.4 enhances data movement between Postgres and Parquet for analytics and lakehouse architectures.
- Lightweight and performant, leveraging Postgres native COPY command.
- Community contributions and feedback encouraged via GitHub.