{"repo":"miku/esbulk","free":true,"listed":false,"github":"https://github.com/miku/esbulk","clone":"git clone https://github.com/miku/esbulk.git","description":"Bulk indexing command line tool for elasticsearch.","language":"Go","stars":287,"topics":["elasticsearch","indexing","code4lib","hacktoberfest"],"license":"GPL-3.0","category":"cli-tools","readme_excerpt":"esbulk ====== Fast parallel command line bulk loading utility for elasticsearch. Data is read from a newline delimited JSON file or stdin and indexed into elasticsearch in bulk and in parallel. The shortest command would be: Caveat: If indexing pressure on the bulk API is too high (dozens or hundreds of parallel workers, large batch sizes, depending on you setup), esbulk will halt and report an error: Please note that, in such a case, some documents are indexed and some are not. Your index will be in an inconsistent state, since there is no transactional bracket around the indexing process. However, using defaults (parallelism: number of cores) on a single node setup will just work. For larger clusters, increase the number of workers until you see full CPU utilization. After that, more workers won't buy any more speed. Currently, esbulk is tested against elasticsearch versions 5, 6, 7 and 8 using testcontainers. Originally written for Leipzig University Library, project finc. Installation ------------ $ go install github.com/miku/esbulk/cmd/esbulk@latest For deb or rpm packages, see: https://github.com/miku/esbulk/releases Usage ----- $ esbulk -h Usage of esbulk: -0 set the number of replicas to 0 during indexing -apikey string set the encoded ES api key (mutually exclusive with -u) -c string create index mappings, settings, aliases, https://is.gd/3zszeu -cpuprofile string write cpu profile to file -id string name of field to use as id field, by default ids are autogenerated ","default_branch":null,"files":null,"tree":[],"storefront":"/r/miku","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/miku/esbulk/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}