{"repo":"scribd/objinsync","free":true,"listed":false,"github":"https://github.com/scribd/objinsync","clone":"git clone https://github.com/scribd/objinsync.git","description":"Continuously synchronize directories from remote object store to local filesystem","language":"Go","stars":109,"topics":["s3","airflow","cplat"],"license":"MIT","category":"data-pipelines","readme_excerpt":"ObjInSync ========= Daemon to continuously and incrementally synchronize a directory from remote object store to a local directory. Usage ----- When running in daemon mode (without --once flag), a health check endpoint is served at :8087/health and a prometheus metrics endpoint is served at :8087/metrics . You can use --status-addr to override the binding address. Objinsync also comes with builtin Sentry integration. To enable it, set the SENTRY DSN environment variable. You can also run objinsync in pull once mode, which behaves just like aws s3 sync : To use with Minio instead of S3, you can set --s3-endpoint and --disable-ssl flags for pull command as you see fit. The -i or --interval flags allows to configure the pull time interval, which is 5 seconds by default: --- Enable debug logs by setting the DEBUG environment variable DEBUG=1 objinsync pull ... Installation ------------ Simply download the prebuilt single binary from release page or use go get command: Pre-built docker images are available at https://github.com/orgs/scribd/packages/container/package/objinsync. Development ------------ Run tests Run from source To cut a release, push tag to remote in the format of vx.x.x .","default_branch":null,"files":null,"tree":[],"storefront":"/r/scribd","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/scribd/objinsync/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}