{"repo":"cavv-dev/crocdb-db","free":true,"listed":false,"github":"https://github.com/cavv-dev/crocdb-db","clone":"git clone https://github.com/cavv-dev/crocdb-db.git","description":"Crocdb Database","language":"Python","stars":26,"topics":["crocdb","roms","database"],"license":"GPL-3.0","category":"databases-storage","readme_excerpt":"Crocdb Database This codebase includes modules and scripts used for creating the ROMs database utilized by Crocdb. Codebase understanding The code is built with modularity in mind. The main groups of modules are: - scrapers - Used for scraping the sources URLs responses. Ideally each module represents a different host, like myrient.py for Myrient and internet archive.py for Internet Archive. - parsers - Used for parsing entries scraped by the scrapers modules, enriching each entry with appropriate information. Main scripts - make.py - Initializes the database and starts processing the sources. Can use cached responses from sources URLs by passing --use-cached , useful for testing purposes. - workflow.py - Initiates the workflow needed for updating additional data needed by scrapers/parsers and starting the database creation. Available scraping/parsing modules Scrapers - myrient - Indexes from Myrient. - internet archive - Indexes from Internet Archive. Log in credentials have to be specified in internet archive creds.json to allow for restricted content to be scraped. - nopaystation - TSV files from NoPayStation. - mariocube - Indexes from MarioCube. Parsers - libretro - Adds ROM IDs and Box art URLs to entries that are listed in the Libretro DAT files by checking title correspondences. Flags: None - no intro - Performs regions extraction from the ROM's title and refactors it by excluding unnecessary content. Designed to work with entries that have titles following the No-Int","default_branch":null,"files":null,"tree":[],"storefront":"/r/cavv-dev","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cavv-dev/crocdb-db/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}