{"repo":"zeromicro/cds","free":true,"listed":false,"github":"https://github.com/zeromicro/cds","clone":"git clone https://github.com/zeromicro/cds.git","description":"Data syncing in golang for ClickHouse.","language":"Go","stars":979,"topics":["go","golang","clickhouse","bigdata","kafka-consumer"],"license":"MIT","category":"data-pipelines","readme_excerpt":"ClickHouse Data Synchromesh Data syncing in golang for ClickHouse. based on go-zero ARCH Data workflow of a typical data warehouse architecture Design of Data Sync Automatically synchronizing data from MySQL/MongoDB data source to ClickHouse cluster in real time(< 2min). start up After the build , pay attention to check if any container exits abnormally. click http://localhost:3414/cds.html to accessing the web interface. using username and password below to login create table in ClickHouse for syncing MySQL(or MongoDB) data Chose \"create table\" tab One-time full data synchronization: Chose \"full sync\" tab check task status Refresh the page Confirm data in Clickhouse Turn on real-time incremental synchronization Take mysql as an example chose \"Connector\" tab chose \"Incremental Sync\" tab refresh page Verify incremental update Execute the initialization database script again, you can insert 100000 rows of data again. Verify the incremental data of mysql in clickhouse： clean up To clean up all the docker containers started above and restore the initial state, you can ： only clean data model in clickhouse CDS中ClickHouse使用的建表方案 help 提问的智慧 如何有效的报告bug --- if you like this project and want to support it，please star 🤝","default_branch":null,"files":null,"tree":[],"storefront":"/r/zeromicro","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/zeromicro/cds/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}