{"repo":"Ensembl/ensembl-hive","free":true,"listed":false,"github":"https://github.com/Ensembl/ensembl-hive","clone":"git clone https://github.com/Ensembl/ensembl-hive.git","description":"EnsEMBL Hive - a system for creating and running pipelines on a distributed compute resource","language":"Perl","stars":55,"topics":["ensembl","workflow-management-system","perl","python","mysql","sqlite","postgresql","high-performance-computing","pipeline","ehive"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"eHive ===== eHive is a system for running computation pipelines on distributed computing resources - clusters, farms or grids. The name comes from the way pipelines are processed by a swarm of autonomous agents. [!IMPORTANT] As per eHive version 2.7.0, we changed the preferred/default eHive meadow from LSF to SLURM . Also, because our pipelines are transitioning to SLURM , we have decided to deprecate all the meadows other than SLURM and Local , and not to support them anymore with immediate effect. For details about the schedulers, please see Grid scheduler and Meadows below in this README. Please, do not hesitate to contact us, should this be a problem for you. Available documentation ----------------------- The main entry point is available online in the user manual, from where it can be downloaded for offline access. eHive in a nutshell ------------------- Blackboard, Jobs and Workers In the centre of each running pipeline is a database that acts as a blackboard with individual tasks to be run. These tasks (we call them Jobs) are claimed and processed by \"Worker bees\" or just Workers - autonomous processes that are continuously running on the compute farm and connect to the pipeline database to report about the progress of Jobs or claim some more. When a Worker discovers that its predefined time is up or that there are no more Jobs to do, it claims no more Jobs and exits the compute farm freeing the resources. Beekeeper A separate Beekeeper process makes sure there are al","default_branch":null,"files":null,"tree":[],"storefront":"/r/Ensembl","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Ensembl/ensembl-hive/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}