{"repo":"aborruso/scrape-cli","free":true,"listed":false,"github":"https://github.com/aborruso/scrape-cli","clone":"git clone https://github.com/aborruso/scrape-cli.git","description":"Extract HTML elements from the command line using CSS selectors or XPath. Pipe-friendly Python CLI.","language":"HTML","stars":29,"topics":["cli","command-line-tool","css-selectors","data-extraction","html","lxml","python","scraping","web-scraping","xpath"],"license":null,"category":"cli-tools","readme_excerpt":"scrape cli It's a command-line tool to extract HTML elements using an XPath query or CSS3 selector . It's based on the great and simple scraping tool written by Jeroen Janssens . - How does it work? - How to use it in Linux - Note on building it Installation You can install scrape-cli using several methods: Using pipx (recommended for CLI tools) Using uv (modern Python package manager) Using pip Or install from source: Requirements - Python =3.6 - requests - lxml - cssselect How does it work? Using the Test HTML File In the resources directory you'll find a test.html file that you can use to test various scraping scenarios. Note : You can also test directly from the URL without cloning the repository: Here are some examples: 1. Extract all table data: 2. Get all list items: 3. Extract specific attributes: 4. Check if an element exists: 5. Extract nested elements: 6. Get elements with specific attributes: 7. Additional XPath examples: General Usage Examples A CSS selector query like this Note: When using both -b and -e options together, they must be specified in the order -be (body first, then expression). Using -eb will not work correctly. or an XPATH query like this one: gives you back: Text Extraction You can extract only the text content (without HTML tags) using the -t option, which is particularly useful for LLMs and text processing: The -t option automatically excludes text from and tags and cleans up whitespace for better readability. JSON Output Use the -j / --json fl","default_branch":null,"files":null,"tree":[],"storefront":"/r/aborruso","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/aborruso/scrape-cli/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}