{"repo":"ndgigliotti/shopify-spy","free":true,"listed":false,"github":"https://github.com/ndgigliotti/shopify-spy","clone":"git clone https://github.com/ndgigliotti/shopify-spy.git","description":"Extract structured data from Shopify websites.","language":"Python","stars":113,"topics":["spider","scrapy","shopify","ecommerce","dropshipping","cli","web-scraping"],"license":"MIT","category":"ecommerce","readme_excerpt":"Shopify Spy Shopify Spy is a command-line tool for scraping product and collection data from ecommerce stores. Built on Scrapy, it supports Shopify and WooCommerce stores out of the box. To find Shopify stores to scrape, try searching Google with site:myshopify.com . Installation pipx and uv tool install CLI tools in isolated environments, so they won't conflict with other Python projects: Or install with pip if you want it in a specific virtual environment: Requires Python 3.10+. Quick Start Results are saved as JSONL in the output directory (default: ./output ). Use --format to choose JSON, CSV, or XML instead. A live item counter is displayed during scraping. Features - Automatic user-agent rotation. Requests use a random browser user-agent. If a server responds with 403, the tool swaps to a different UA and retries. - Failure diagnostics. When a scrape returns 0 items, the tool explains why (403, 404, robots.txt, timeout) and suggests flags to try. - Log files. Scrapy's verbose output goes to a log file, keeping the terminal clean. Use --verbose to also print to the terminal. - Status JSON. Each run writes a machine-readable status file with per-URL results, errors, and timing. - Bail timeout. Automatically aborts if no items are scraped within --bail seconds (default: 30). - Coding-agent friendly. Designed for use by AI coding agents (Claude Code, Cursor, etc.): - --peek pipes a single item to stdout for quick schema inspection - Non-zero exit codes on failure with diagn","default_branch":null,"files":null,"tree":[],"storefront":"/r/ndgigliotti","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ndgigliotti/shopify-spy/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}