{"repo":"trudi-group/ipfs-crawler","free":true,"listed":false,"github":"https://github.com/trudi-group/ipfs-crawler","clone":"git clone https://github.com/trudi-group/ipfs-crawler.git","description":"A crawler for the IPFS network, code for our paper (https://arxiv.org/abs/2002.07747). Also holds scripts to evaluate the obtained data and make similar plots as in the paper.","language":"Go","stars":70,"topics":["ipfs","ipfs-network","libp2p","kademlia-dht","crawler"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Libp2p-Crawler A crawler for the Kademlia-part of various libp2p networks. For more details, see our paper. If you use our work, please cite our papers : Sebastian A. Henningsen, Martin Florian, Sebastian Rust, Björn Scheuermann: Mapping the Interplanetary Filesystem. Networking 2020 : 289-297\\ [[BibTex]](https://dblp.uni-trier.de/rec/conf/networking/HenningsenFR020.html?view=bibtex) Sebastian A. Henningsen, Sebastian Rust, Martin Florian, Björn Scheuermann: Crawling the IPFS Network. Networking 2020 : 679-680\\ [[BibTex]](https://dblp.uni-trier.de/rec/conf/networking/HenningsenRF020.html?view=bibtex) For a Live Version of the crawler results, check out our Periodic Measurements of the IPFS Network Building You can build this in a containerized environment. This will build on Debian Bullseye and extract the compiled binary to out/ : This is the preferred way of compilation. You can also manually compile the crawler. This will need an older version of Go installed, since the most recent version is usually not supported by the QUIC implementation. Usage To crawl the network once, execute the crawler with the corresponding config file: One crawl will take 5-10 minutes, depending on your machine. Docker The image executes dist/docker entrypoint.sh by default, which will set the environment variables and launch the crawler with all arguments provided to it. This loads a config file located at /libp2p-crawler/config.yaml in the image. You can thus override the executed config by mou","default_branch":null,"files":null,"tree":[],"storefront":"/r/trudi-group","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/trudi-group/ipfs-crawler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}