{"repo":"medialab/minet","free":true,"listed":false,"github":"https://github.com/medialab/minet","clone":"git clone https://github.com/medialab/minet.git","description":"A webmining CLI tool & library for python.","language":"Python","stars":372,"topics":["webmining","cli","python"],"license":"GPL-3.0","category":"cli-tools","readme_excerpt":"minet is a webmining command line tool & library for python ( = 3.8) that can be used to collect and extract data from a large variety of web sources such as raw webpages, Facebook, YouTube, Twitter, Media Cloud etc. It adopts a very simple approach to various webmining problems by letting you perform a wide array of tasks from the comfort of the command line. No database needed: raw CSV files should be sufficient to do most of the work. In addition, minet also exposes its high-level programmatic interface as a python library so you remain free to use its utilities to suit your use-cases better. minet is developed by médialab SciencesPo research engineers and is the consolidation of more than a decade of webmining practices targeted at social sciences. As such, it has been designed to be: 1. low-tech , as it requires minimal resources such as memory, CPUs or hard drive space and should be able to work on any low-cost PC. 2. fault-tolerant , as it is able to recover when network is bad and retry HTTP calls when suitable. What's more, most of minet commands can be resumed if aborted and are designed to run for a long time (think days or months) without leaking memory. 3. unix-compliant , as it can be piped easily and know how to work with the usual streams. Shortcuts : Command line documentation, Python library documentation. How to cite? minet is published on Zenodo as 10.5281/zenodo.4564399. You can cite it thusly: Guillaume Plique, Pauline Breteau, Jules Farjas, Héloïse Thér","default_branch":null,"files":null,"tree":[],"storefront":"/r/medialab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/medialab/minet/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}