{"repo":"jamessanford/remote-tsdb-clickhouse","free":true,"listed":false,"github":"https://github.com/jamessanford/remote-tsdb-clickhouse","clone":"git clone https://github.com/jamessanford/remote-tsdb-clickhouse.git","description":"A remote writer/reader for Prometheus that stores TSDB data in ClickHouse","language":"Go","stars":46,"topics":["clickhouse","prometheus","remote-write"],"license":"Apache-2.0","category":"analytics","readme_excerpt":"remote-tsdb-clickhouse stores timeseries data in ClickHouse. Implements both Prometheus remote writer (to store metrics) and Prometheus remote reader (use metrics from ClickHouse directly in Prometheus) Install Create destination table Use clickhouse client to create this table: This works well with over 100 billion metrics, even when searching by label, although cardinality of my dataset is very low at 16032 unique metrics+labels. Including label values, it takes approximately 1 byte per value for my dataset (1 gigabyte per billion metrics) The labelset skip index has a high granularity to try and keep the planning cost low as it may not be useful for typical queries. Your mileage may vary. Storing and indexing the labels array directly is a naive implementation, setups with millions of unique metrics will need more advanced setups with a label hash or fingerprint and a separate lookup table. Configure Prometheus remote writer In your prometheus.yaml : ClickHouse prefers fewer writes with more samples per write. Above a certain rate you may need to adjust Prometheus capacity and max samples per send as per Prometheus Remote Write Tuning if you see \"Too many parts\" errors or prometheus remote storage samples pending keeps growing. Configure Prometheus remote reader In your prometheus.yaml : Query data with Prometheus The above configuration will use remote-tsdb-clickhouse to backfill data not present in Prometheus. If you'd like to query remote-tsdb-clickhouse immediately, co","default_branch":null,"files":null,"tree":[],"storefront":"/r/jamessanford","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jamessanford/remote-tsdb-clickhouse/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}