{"repo":"eyeseast/feed-to-sqlite","free":true,"listed":false,"github":"https://github.com/eyeseast/feed-to-sqlite","clone":"git clone https://github.com/eyeseast/feed-to-sqlite.git","description":"Save an RSS or ATOM feed to a SQLite database","language":"Python","stars":58,"topics":["atom-feed","sqlite-database","rss","datasette","sqlite"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"feed-to-sqlite Download an RSS or Atom feed and save it to a SQLite database. This is meant to work well with datasette. Installation CLI Usage Let's grab the ATOM feeds for items I've shared on NewsBlur and my instapaper favorites save each its own table. This will use a SQLite database called feeds.db , creating it if necessary. By default, each feed gets its own table, named based on a slugified version of the feed's title. To load all items from multiple feeds into a common (or pre-existing) table, pass a --table argument: That will put all items in a table called links . Each feed also creates an entry in a feeds table containing top-level metadata for each feed. Each item will have a foreign key to the originating feed. This is especially useful if combining feeds into a shared table. Python API One function, ingest feed , does most of the work here. The following will create a database called feeds.db and download my NewsBlur shared items into a new table called links . Transforming data on ingest When working in Python directly, it's possible to pass in a function to transform rows before they're saved to the database. The normalize argument to ingest feed is a function that will be called on each feed item, useful for fixing links or doing additional work. It's signature is normalize(table, entry, feed details, client) : - table is a SQLite table (from sqlite-utils) - entry is one feed item, as a dictionary - feed details is a dictionary of top-level feed information","default_branch":null,"files":null,"tree":[],"storefront":"/r/eyeseast","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/eyeseast/feed-to-sqlite/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}