{"repo":"socius-org/RedditHarbor","free":true,"listed":false,"github":"https://github.com/socius-org/RedditHarbor","clone":"git clone https://github.com/socius-org/RedditHarbor.git","description":"Ethical, legal, and effortless extraction of Reddit data in your database","language":"TypeScript","stars":98,"topics":["database","reddit","reddit-api","reddit-crawler","reddit-scraper","supabase-db"],"license":"MIT","category":"databases-storage","readme_excerpt":"RedditHarbor RedditHarbor simplifies collecting Reddit data and saving it to a database. It removes the complexity of working with APIs, letting you easily build a \"harbor\" of Reddit data for analysis. Introduction Social media data from platforms like Reddit contains rich insights into human behaviour and beliefs. However, collecting and storing this data requires dealing with complex APIs. RedditHarbor streamlines this entire process so you can focus on your research. In plain language: - ✨ Comprehensive API Data Collection : Gather Reddit submissions, comments, and user profiles directly from the official data API. - 🔒 Privacy-Preserving : Anonymise PII to protect user privacy and meet ethical/IRB standards. - 📦 Controlled Data Storage : Store collected data in your own secure database for accessibility and organisation. - 📈 Highly Scalable : Handle massive datasets with millions of rows through efficient pagination. - 🕹️ Configurable Collection : Tailor data gathering to your specific needs via adjustable parameters. - 📂 Analysis-Ready Exports : Export to CSV, JSON, JPEG for seamless integration with analysis tools. - 🔄 Temporal Metric Tracking : Regularly update post metrics like scores, upvote ratios, awards over time - unlike static snapshot databases. - ⚡ Smart Update Intervals : Automatically adjust update frequency based on dataset size for optimised API efficiency. Minimum coding required after the initial setup! The tool is designed specifically for research","default_branch":null,"files":null,"tree":[],"storefront":"/r/socius-org","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/socius-org/RedditHarbor/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}