{"repo":"Human-Centric-Machine-Learning/stackexchange-dump-to-postgres","free":true,"listed":false,"github":"https://github.com/Human-Centric-Machine-Learning/stackexchange-dump-to-postgres","clone":"git clone https://github.com/Human-Centric-Machine-Learning/stackexchange-dump-to-postgres.git","description":"Python scripts to import StackExchange data dump into Postgres DB.","language":"Python","stars":92,"topics":["stackexchange-dump","postgres","python","database","data-dump","stackoverflow-data"],"license":"MIT","category":"databases-storage","readme_excerpt":"StackOverflow data to postgres This is a quick script to move the Stackoverflow data from the StackExchange data dump (Sept '14) to a Postgres SQL database. Schema hints are taken from a post on Meta.StackExchange and from StackExchange Data Explorer. Quickstart Install requirements, create a new database (e.g. beerSO below), and use load into pg.py script: This will download compressed files from archive.org and load all the tables at once. Advanced Usage You can use a custom database name as well. Make sure to explicitly give it while executing the script later. Each table data is archived in an XML file. Available tables varies accross history. load into pg.py knows how to handle the following tables: - Badges . - Posts . - Tags (not present in earliest dumps). - Users . - Votes . - PostLinks . - PostHistory . - Comments . You can download manually the files to the folder from where the program is executed: Badges.xml , Votes.xml , Posts.xml , Users.xml , Tags.xml . In some old dumps, the cases in the filenames are different. Then load each file with e.g. python load into pg.py -t Badges . After all the initial tables have been created: For some additional indexes and tables, you can also execute the the following; If you give a schema name using the -n switch, all the tables will be moved to the given schema. This schema will be created in the script. The paths are not changed in the final scripts sql/final post.sql and sql/optional post.sql . To run them, first set the s","default_branch":null,"files":null,"tree":[],"storefront":"/r/Human-Centric-Machine-Learning","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Human-Centric-Machine-Learning/stackexchange-dump-to-postgres/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}