{"repo":"elliotgao2/toapi","free":true,"listed":false,"github":"https://github.com/elliotgao2/toapi","clone":"git clone https://github.com/elliotgao2/toapi.git","description":"Every web site provides APIs.","language":"Python","stars":3555,"topics":["html","json","api","python","web","spider","crawler","flask","toapi"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"toapi Turn any website into a JSON API — declaratively. toapi lets you point at a web page, declare the fields you want with CSS selectors, and get back a clean JSON API. No crawler to babysit, no database to maintain — pages are fetched and parsed on demand, with built‑in caching. Install Requires Python 3.10+. Quickstart Run it: Then visit and you get: How it works 1. Route — @api.route(\"/posts\", \"/news\") maps your API path to a source URL. 2. Fetch — pages are fetched with requests (or a headless browser if you pass browser= ) and cached in memory. 3. Parse — each Item extracts fields with CSS selectors via htmlparsing . 4. Serve — Flask returns the result as JSON; subsequent calls hit the cache. Features - Declarative — describe data, not scraping logic. - Routes — map clean API paths to messy source URLs with {param} placeholders. - Multi-site — merge several websites behind one API. - Cleaning hooks — define clean methods to post-process values. - Caching — pages and parsed results are cached automatically. - Headless browser — pass Api(browser=\"/path/to/geckodriver\") for JS-heavy sites. Cleaning values Add a clean method on the Item to transform a value before it's returned: Development We use uv for packaging and ruff for lint + format. Pre-commit hooks keep both clean: Contributing Pull requests are welcome. For non-trivial changes, please open an issue first to discuss what you'd like to change. Make sure uv run pytest and uv run ruff check . pass before submitting.","default_branch":null,"files":null,"tree":[],"storefront":"/r/elliotgao2","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/elliotgao2/toapi/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}