{"repo":"ropensci/arkdb","free":true,"listed":false,"github":"https://github.com/ropensci/arkdb","clone":"git clone https://github.com/ropensci/arkdb.git","description":"Archive and unarchive databases as flat text files","language":"R","stars":80,"topics":["database","archiving","r","rstats","dbi","peer-reviewed","r-package"],"license":null,"category":"databases-storage","readme_excerpt":"arkdb 🗞️ PLEASE NOTE 🗞️ It is now easy to accomplish arkdb 's tasks of moving between larger-than-ram csv/parquet/etc flatfiles and SQL databases using duckdb . Even better, duckdb supports larger-than-ram DBI operations directly on flatfiles without even importing, better compatibility with a wide range of SQL databases (see extensions), and better performance. See the R package, duckdbfs for an easy way to use duckdb from R. --- The goal of arkdb is to provide a convenient way to move data from large compressed text files (tsv, csv, etc) into any DBI-compliant database connection (e.g. MYSQL, Postgres, SQLite; see DBI), and move tables out of such databases into text files. The key feature of arkdb is that files are moved between databases and text files in chunks of a fixed size, allowing the package functions to work with tables that would be much too large to read into memory all at once. There is also functionality for filtering and applying transformation to data as it is extracted from the database. The arkdb package is easily extended to use custom read and write methods allowing you to dictate your own output formats. See R/streamable table.R for examples that include using: - Base c/tsv - Apache arrow’s parquet - The readr package for c/tsv Links - A more detailed introduction to package design and use can be found in the package Vignette - Online versions of package documentation Installation You can install arkdb from GitHub with: Basic use Creating an archive ","default_branch":null,"files":null,"tree":[],"storefront":"/r/ropensci","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ropensci/arkdb/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}