{"repo":"CrunchyData/pg_parquet","free":true,"listed":false,"github":"https://github.com/CrunchyData/pg_parquet","clone":"git clone https://github.com/CrunchyData/pg_parquet.git","description":"Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL","language":"Rust","stars":683,"topics":["columnar","data-ingestion","data-migration","parquet","postgresql","azure-storage","google-cloud-storage","https","s3"],"license":null,"category":"databases-storage","readme_excerpt":"pg parquet Copy from/to Parquet files in PostgreSQL! pg parquet is a PostgreSQL extension that allows you to read and write Parquet files, which are located in S3 , Azure Blob Storage , Google Cloud Storage , http(s) endpoints or file system , from PostgreSQL via COPY TO/FROM commands. It depends on Apache Arrow project to read and write Parquet files and pgrx project to extend PostgreSQL's COPY command. Quick Reference - Installation From Source - Usage - Copy FROM/TO Parquet files TO/FROM Postgres tables - COPY FROM/TO Parquet stdin/stdout TO/FROM Postgres tables) - COPY FROM/TO Parquet program stream TO/FROM Postgres tables) - Inspect Parquet schema - Inspect Parquet metadata - Inspect Parquet column statistics - List and read Parquet files from uri pattern - Object Store Support - Copy Options - Configuration - Supported Types - Nested Types - Postgres Support Matrix Installation From Source After installing Postgres , you need to set up rustup , cargo-pgrx to build the extension. Usage There are mainly 3 things that you can do with pg parquet : 1. You can export Postgres tables/queries to Parquet files, stdin/stdout or a program's stream, 2. You can ingest data from Parquet files to Postgres tables, 3. You can inspect the schema and metadata of Parquet files. COPY from/to Parquet files to/from Postgres tables You can use PostgreSQL's COPY command to read and write from/to Parquet files. Below is an example of how to write a PostgreSQL table, with complex types, into a Pa","default_branch":null,"files":null,"tree":[],"storefront":"/r/CrunchyData","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/CrunchyData/pg_parquet/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}