{"repo":"dhrp/moulin","free":true,"listed":false,"github":"https://github.com/dhrp/moulin","clone":"git clone https://github.com/dhrp/moulin.git","description":"A reliable queue for batch processing written in Go and backed by Redis.","language":"Go","stars":16,"topics":["redis","grpc","go","batch-processing"],"license":null,"category":"databases-storage","readme_excerpt":"Moulin 'Moulin' is 'Mill' in French. A Dutch saying goes: wie het eerst komt, wie het eerst maalt . Who brings their wheat to the mill first, is the first to get his/her wheat milled, and is the first to eat. For information how it works, read here Running the service Run the server First make sure you have Redis running. For example like so: Then start the moulin server. In this example we link to the redis container that we just started. Specify the REDIS ADDRESS, to point to a Redis server (defaults to localhost) Run the client The client is also built into the docker image. You can run it with Here we tell use moulin-cli to listen to a queue 'queue1' and execute each command that comes from the queue. Using Moulin How to use it Moulin primarily meant to simply execute a given task on the shell; and as such it is language agnostic. Let's say you have a python script \"download files.py\" which takes one argument, a URL to download. You can create a task like: You can then also create a worker like so: It will listen on a socket from the moulin server, pop a task from the queue and processes it. When no more tasks are available it will wait for the next to show up. Reliability Once a Moulin worker takes a task from the queue it moves to a 'in process' queue, and only moves to \"succeeded\" if the child process exits with error code 0. The worker is also expected to send a regular heartbeat to the server to make sure the process didn't die unexpectedly. After 5 minutes of no hea","default_branch":null,"files":null,"tree":[],"storefront":"/r/dhrp","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/dhrp/moulin/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}