{"repo":"anki-code/xontrib-pipeliner","free":true,"listed":false,"github":"https://github.com/anki-code/xontrib-pipeliner","clone":"git clone https://github.com/anki-code/xontrib-pipeliner.git","description":"Let your pipe lines flow thru the Python code in Xonsh shell.","language":"Python","stars":63,"topics":["xonsh","xontrib","pipe","pipeline","pipelines","python","shell","data-engineering","data-engineering-pipeline"],"license":"BSD-2-Clause","category":"data-pipelines","readme_excerpt":"Easily process the lines using pipes in Xonsh shell . Multicore processing supported. If you like the idea of pipeliner click ⭐ on the repo and tweet . Install Usage Let your pipe lines flow thru the Python code: Experimental: ppl is to run multicore pl . It tested mostly on Linux. See \"Known issues\" section. Examples Presets You can set your own presets: Lambda string There are two variables available in lambda expression: line from pipe. num of the line starts with 0. Python way to line modification Line number Ignore line Splitting Imports Arrays Python head Variables and operations chaining Expression is a lambda function so using variables and operations chaining since Python 3.8+ are available by trick with the walrus operator and the list: Execute command with the line Note! If you do the operations with files (i.e. pl \"execx(f'mv {line} prefix-{line}')\" ) you could catch TypeError: an integer is required error that relates to wrong access rights to files. Fix it with chmod and chown before pipelining. Wrap pipeliner to get your own magic Add your most useful solutions to xontrib-pipeliner. PRs are welcome! Experimental Syntax highlighting using xonsh prompt If you're using xonsh prompt and want to use pipeliner with syntax highlighting instead of string try @!() : Multicore pipelining By default pipeliner works using one CPU core. To use them all in parallel try ppl command: Note! The order of result lines is unpredictable because lines will be processed in parallel. ","default_branch":null,"files":null,"tree":[],"storefront":"/r/anki-code","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/anki-code/xontrib-pipeliner/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}