{"repo":"DBeath/feedsearch-crawler","free":true,"listed":false,"github":"https://github.com/DBeath/feedsearch-crawler","clone":"git clone https://github.com/DBeath/feedsearch-crawler.git","description":"Crawl sites for RSS, Atom, and JSON feeds.","language":"Python","stars":97,"topics":["rss","crawler","feeds","python3","scraping","python","crawling","feedsearch-crawler","asyncio","aiohttp"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Feedsearch Crawler feedsearch-crawler is a Python library for discovering RSS, Atom), and JSON feeds on websites. About This is a library package designed to be integrated into other Python applications. It provides a simple API for feed discovery that can be embedded into web scrapers, content aggregators, RSS readers, or API services. It is a continuation of my work on Feedsearch, which is itself a continuation of the work done by Dan Foreman-Mackey on Feedfinder2, which in turn is based on feedfinder - originally written by Mark Pilgrim) and subsequently maintained by Aaron Swartz until his untimely death. Feedsearch Crawler differs from previous versions in that it is now built as an asynchronous Web crawler using asyncio and aiohttp, allowing much more rapid scanning of potential feed URLs. Real-World Usage An implementation using this library to provide a public Feed Search API is available at Pull requests and suggestions are welcome. Installation The library is available on PyPI: Requirements: - Python 3.12 or higher - No additional system dependencies Usage Feedsearch Crawler is called with the single function search : If you are already running in an asyncio event loop, then you can import and await search async instead. The search function is only a wrapper that runs search async in a new asyncio event loop. A search will always return a list of FeedInfo objects, each of which will always have a url property, which is a URL object that can be decoded to a string wi","default_branch":null,"files":null,"tree":[],"storefront":"/r/DBeath","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/DBeath/feedsearch-crawler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}