{"repo":"tobymao/saq","free":true,"listed":false,"github":"https://github.com/tobymao/saq","clone":"git clone https://github.com/tobymao/saq.git","description":"Simple Async Queues","language":"Python","stars":879,"topics":["redis","queue","asyncio","python","message-queue"],"license":"MIT","category":"databases-storage","readme_excerpt":"SAQ SAQ (Simple Async Queue) is a simple and performant job queueing framework built on top of asyncio and redis or postgres. It can be used for processing background jobs with workers. For example, you could use SAQ to schedule emails, execute long queries, or do expensive data analysis. Documentation It uses redis-py = 4.2. It is similar to RQ and heavily inspired by ARQ. Unlike RQ, it is async and thus significantly faster if your jobs are async. Even if they are not, SAQ is still considerably faster due to lower overhead. SAQ optionally comes with a simple UI for monitor workers and jobs. Install Usage Example To start the worker, assuming the previous is available in the python path Note: module.file.settings can also be a callable returning the settings dictionary. To enqueue jobs Demo Start the worker Navigate to the web ui Enqueue jobs Comparison to ARQ SAQ is heavily inspired by ARQ but has several enhancements. 1. Avoids polling by leveraging BLMOVE or RPOPLPUSH and NOTIFY 1. SAQ has much lower latency than ARQ, with delays of < 5ms. ARQ's default polling frequency is 0.5 seconds 2. SAQ is up to 8x faster than ARQ 2. Web interface for monitoring queues and workers 3. Heartbeat monitor for abandoned jobs 4. More robust failure handling 1. Storage of stack traces 2. Sweeping stuck jobs 3. Handling of cancelled jobs different from failed jobs (machine redeployments) 5. Before and after job hooks 6. Easily run multiple workers to leverage more cores Development","default_branch":null,"files":null,"tree":[],"storefront":"/r/tobymao","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/tobymao/saq/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}