{"repo":"retail-ai-inc/sync","free":true,"listed":false,"github":"https://github.com/retail-ai-inc/sync","clone":"git clone https://github.com/retail-ai-inc/sync.git","description":"Sync your production database (PII proof) to a secondary database (cold storage) instance for data scientists, data synthesization, data partitioning or other purposes. Supports MongoDB, MySQL, PostgreSQL, Redis with real-time sync. #hot-storage #cold-storage","language":"Go","stars":64,"topics":["change-streams","data-migration","database-sync","database-synchronization","golang","mongodb","mongodb-sync","mysql","postgresql","real-time"],"license":"MIT","category":"databases-storage","readme_excerpt":"Sync Sync is a real-time, one-way, and PII-proof database backup tool written in Golang to synchronize NOSQL and SQL data to standalone instances, GCS, using a UI and workflow engine for Data scientists, auditing, and various other purposes. Sync can synchronize MongoDB or SQL data from a MongoDB replica set or sharded cluster , or production SQL instance to a standalone instance , supports initial and incremental synchronization, including indexes with change stream monitoring. [!NOTE] - Sync now supports MongoDB, MySQL, PostgreSQL, MariaDB, and Redis. Next, Sync will support Elasticsearch. - Sync is PII proof, meaning you can select specific records, keys, columns from your source databases, and sync them into target databases using one-way encryption or masking methods. What is the problem - Let's assume you have multiple tech teams and stakeholders. Different teams have different requirements for analyzing the production data independently. However, the tech team doesn't want to allow all these stakeholders direct access to the production databases due to security and stability issues. - Another use case is to support your hot storage and cold storage policies. For example, allowing your smartphone app users to access only the latest 1 year of data from the hot storage, and serve the rest of the data on a request basis from cold storage. A simple one-way solution Create standalone databases outside of your production database servers with the same name as the production d","default_branch":null,"files":null,"tree":[],"storefront":"/r/retail-ai-inc","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/retail-ai-inc/sync/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}