{"repo":"cpg314/polarhouse","free":true,"listed":false,"github":"https://github.com/cpg314/polarhouse","clone":"git clone https://github.com/cpg314/polarhouse.git","description":"Interoperability between Polars and Clickhouse","language":"Rust","stars":14,"topics":["apache-arrow","clickhouse","polars","rust"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"Polarhouse connects together - Polars Dataframes (backed by the Apache Arrow columnar format) - the Clickhouse columnar database. More specifically, it allows: - inserting Polars Dataframes into Clickhouse tables (and creating these if necessary). - and vice-versa retrieving Clickhouse query results as Polars Dataframes. Polarhouse uses the native TCP Clickhouse protocol via the klickhouse crate. It maps the Polars and Clickhouse types, and builds Polars Series (resp. Clickhouse columns) after transforming the data if necessary. This is not yet published on crates.io , as it depends on a fork of the klickhouse crate exposing the Block::read method (see this PR). Polars to Clickhouse Rust Clickhouse to Polars Rust Python Status This is for now only a proof of concept. Alternative solutions - Use the Arrow , ArrowStream or Parquet Clickhouse input/output formats, which can be read and written from Polars. - Write an Arrow Database Connectivity driver for Clickhouse, and use Polars' ADBC support. - Clickhouse to Polars: ConnectorX (uses the MySQL interface). Tests Supported types - [x] Integers - [x] Floating points - [x] Strings - [x] Booleans - [x] Categorical (Polars) / Low cardinality (Clickhouse) - [x] Structs (Polars), which get flattened into Clickhouse, with fields names separated by . - [x] Nullables - [x] Lists (Polars) / Arrays (Clickhouse) - [x] UUIDs (mapped to Strings in Polars) - [ ] Arrays (Polars) - [ ] Tuples - [ ] DateTime - [ ] Time - [ ] Duration - [ ] ...","default_branch":null,"files":null,"tree":[],"storefront":"/r/cpg314","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cpg314/polarhouse/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}