{"repo":"shenwei356/rush","free":true,"listed":false,"github":"https://github.com/shenwei356/rush","clone":"git clone https://github.com/shenwei356/rush.git","description":"A cross-platform command-line tool for executing jobs in parallel","language":"Go","stars":1130,"topics":["bioinformatics","command","cross-platform","execute","golang","parallel","pipeline","shell","windows"],"license":"MIT","category":"dev_tool","readme_excerpt":"# rush -- a cross-platform command-line tool for executing jobs in parallel\n\n[![Built with GoLang](https://img.shields.io/badge/powered_by-go-6362c2.svg?style=flat)](https://golang.org)\n[![Go Report Card](https://goreportcard.com/badge/github.com/shenwei356/rush)](https://goreportcard.com/report/github.com/shenwei356/rush)\n[![Cross-platform](https://img.shields.io/badge/platform-any-ec2eb4.svg?style=flat)](#download)\n[![Latest Version](https://img.shields.io/github/release/shenwei356/rush.svg?style=flat?maxAge=86400)](https://github.com/shenwei356/rush/releases)\n[![Github Releases](https://img.shields.io/github/downloads/shenwei356/rush/latest/total.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases)\n\n`rush` is a tool similar to [GNU parallel](https://www.gnu.org/software/parallel/)\n and [gargs](https://github.com/brentp/gargs).\n `rush` borrows some idea from them and has some unique features,\n  e.g.,\n  supporting custom defined variables,\n  resuming multi-line commands,\n  more advanced embeded replacement strings.\n\nThese features make `rush` suitable for easily and flexibly parallelizing\ncomplex workflows in fields like Bioinformatics (see [examples](#examples)).\n\n\n## Table of Contents\n<!-- START doctoc generated TOC please keep comment here to allow auto update -->\n<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->\n\n- [Features](#features)\n- [Performance](#performance)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Examples](#examples)\n- [Special Cases](#special-cases)\n- [Contributors](#contributors)\n- [Acknowledgements](#acknowledgements)\n- [Contact](#contact)\n- [License](#license)\n\n<!-- END doctoc generated TOC please keep comment here to allow auto update -->\n\n\n## Features\n\nMajor:\n\n- Supporting Linux, OS X and **Windows** (not CygWin)!\n- **Avoid mixed line from multiple processes without loss of performance**,\n  e.g. the first half of a line is from one process\n  and the last half of the line is from another process.\n  (`--line-buffer` in GNU parallel)\n- **Timeout** (`-t`). (`--timeout` in GNU parallel)\n- **Retry** (`-r`). (`--retry-failed --joblog` in GNU parallel)\n- **Safe exit after capturing Ctrl-C** (*not perfect*, you may stop it by typing ctrl-c or closing terminal)\n- **Continue** (`-c`). (`--resume --joblog` in GNU parallel,\n  ***<s/>sut it does not support multi-line commands, which are common in workflow</s>***)\n- **`awk -v` like custom defined variables** (`-v`). (***Using Shell variable in GNU parallel***)\n- **Keeping output in order of input** (`-k`). (Same `-k/--keep-order` in GNU parallel)\n- **Exit on first error(s)** (`-e`). (*not perfect*, you may stop it by typing ctrl-c or closing terminal) (`--halt 2` in GNU parallel) \n- **Settable record delimiter** (`-D`, default `\\n`). (`--recstart` and `--recend` in GNU parallel)\n- **Settable records sending to every command** (`-n`, default `1`). (`-n/--max-args` in GNU parallel)\n- **Settable field delimiter** (`-d`, default `\\s+`). (Same `-d/--delimiter` in GNU parallel)\n- **Practical replacement strings** (like GNU parallel):\n    - `{{}}`, `{}` itself\n    - `{{1,}}`, `{1,}`.\n    - `{#}`, job ID. (Same in GNU parallel)\n    - `{}`, full data. (Same in GNU parallel)\n    - `{n}`, `n`th field in delimiter-delimited data. (Same in GNU parallel)\n    - Directory and file\n        - `{/}`, dirname. (`{//}` in GNU parallel)\n        - `{%}`, basename. (`{/}` in GNU parallel)\n        - `{.}`, remove the last file extension. (Same in GNU parallel)\n        - `{:}`, remove all file extensions (***Not directly supported in GNU parallel***)\n        - `{^suffix}`, remove `suffix` (***Not directly supported in GNU parallel***)\n        - `{@regexp}`, capture submatch using regular expression (***Not directly supported in GNU parallel***).\n           There's a limitation here: curly brackets can't be used in the regular expression.\n    - Combinations\n        - `{%.}`, `{%:}`, basename without extension\n        - `{2.}`, `{2/}`, `{2%.}`, manipulate `n`th field\n        - `{file:}`, `{file:^_1}`, remove all extensions of a preset variable (see below)\n- **Preset variable (macro)**, e.g., `rush -v p={^suffix} 'echo {p}_new_suffix'`,\nwhere `{p}` is replaced with `{^suffix}`. (***Using Shell variable in GNU parallel***)\n\nMinor:\n\n- Dry run (`--dry-run`). (Same in GNU parallel)\n- Trim input data (`--trim`). (Same in GNU parallel)\n- Verbose output (`--verbose`). (Same in GNU parallel)\n\n[Differences between rush and GNU parallel](https://www.gnu.org/software/parallel/parallel_alternatives.html#DIFFERENCES-BETWEEN-Rush-AND-GNU-Parallel) on GNU parallel site.\n\n## Performance\n\nPerformance of `rush` is similar to `gargs`, and they are both slightly faster than `parallel` (Perl) and both slower than `Rust parallel` ([discussion](https://github.com/shenwei356/rush/issues/1)).\n\nNote that speed is not the #.1 target, especially for processes that last long.\n\n\n## Installation\n\n`rush` is implemented in [Go](https://golang.org/) programming language,\n executable binary files **for most popular operating systems** are freely available\n  in [release](https://github.com/shenwei356/rush/releases) page.\n\n#### Method 0: Conda\n\n[Install conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html), then run\n\n    conda install -c conda-forge rush\n\nOr use [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html), which is faster.\n\n    mamba install -c conda-forge rush\n\n#### Method 1: Download binaries\n\n[rush v0.9.0](https://github.com/shenwei356/rush/releases/tag/v0.9.0)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/shenwei356/rush/v0.9.0/total.svg)](https://github.com/shenwei356/rush/releases/tag/v0.9.0)\n\n***Tip: run `rush -V` to check update !!!***\n\nOS     |Arch      |File,                                                                                                                           |Download Count\n:------|:---------|:-------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\nLinux  |32-bit    |[rush_linux_386.tar.gz](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_386.tar.gz)                      |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_linux_386.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_386.tar.gz)\nLinux  |**64-bit**|[**rush_linux_amd64.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_amd64.tar.gz)              |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_linux_amd64.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_amd64.tar.gz)\nLinux  |**arm64** |[**rush_linux_arm64.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_arm64.tar.gz)              |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_linux_arm64.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_linux_arm64.tar.gz)\nOS X   |**64-bit**|[**rush_darwin_amd64.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_darwin_amd64.tar.gz)            |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_darwin_amd64.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_darwin_amd64.tar.gz)\nOS X   |**arm64** |[**rush_darwin_arm64.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_darwin_arm64.tar.gz)            |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_darwin_arm64.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_darwin_arm64.tar.gz)\nWindows|32-bit    |[rush_windows_386.exe.tar.gz](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_windows_386.exe.tar.gz)          |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_windows_386.exe.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_windows_386.exe.tar.gz)\nWindows|**64-bit**|[**rush_windows_amd64.exe.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_windows_amd64.exe.tar.gz)  |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_windows_amd64.exe.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_windows_amd64.exe.tar.gz)\nFreeBSD|32-bit    |[rush_freebsd_386.tar.gz](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_freebsd_386.tar.gz)                  |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_freebsd_386.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_freebsd_386.tar.gz)\nFreeBSD|**64-bit**|[**rush_freebsd_amd64.tar.gz**](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_freebsd_amd64.tar.gz)          |[![Github Releases (by Asset)](https://img.shields.io/github/downloads/shenwei356/rush/latest/rush_freebsd_amd64.tar.gz.svg?maxAge=3600)](https://github.com/shenwei356/rush/releases/download/v0.9.0/rush_freebsd_amd64.tar.gz)\n\n\nJust [download](https://github.com/shenwei356/rush/releases) compressed\nexecutable file of your operating system,\nand decompress it with `tar -zxvf *.tar.gz` command or other tools.\nAnd then:\n\n1. **For Linux-like systems**\n    1. If you have root privilege simply copy it to `/usr/local/bin`:\n\n            sudo cp rush /u","default_branch":"master","files":21,"tree":[".github/FUNDING.yml",".github/issue_template.md",".gitignore",".travis.yml","CHANGELOG.md","LICENSE","README.md","benchmark.py","cobra_helper.go","go.mod","go.sum","helper.go","logging.go","main.go","packaging.sh","process/process.go","process/process_others.go","process/process_windows.go","root.go","string_repl.go","tests/test1.sh"],"storefront":"/r/shenwei356","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/shenwei356/rush/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}