{"repo":"vincentnam/docker_datalake","free":true,"listed":false,"github":"https://github.com/vincentnam/docker_datalake","clone":"git clone https://github.com/vincentnam/docker_datalake.git","description":"Datalake","language":"JavaScript","stars":32,"topics":["datalake","open-source","openstack-swift","apache","apache-airflow","database","software-development","software-design","architecture-design","datamanagement"],"license":null,"category":"databases-storage","readme_excerpt":"The project is now hold on 2 differents repository The official main repository is the IRIT Gitlab repository (https://gitlab.irit.fr/datalake/docker datalake). The secondary repository is the Github initial repository (https://github.com/vincentnam/docker datalake). The github repository is mirrored to follow the progress of the Gitlab repository and to be synchronized. Data lake architecture POC hosted on OSIRIM (https://osirim.irit.fr/site/) This repository is the main working repository for the data lake architecture. The goals are : - Create an opensource architecture for data management based on data lake architecture - Handle any type of data - Handle any volumetry of datas - Make it easy to deploy the architecture - Make an interoperable architecture through the usage of REST API whenever it is possible The development of this architecture will integrate a semantic dimension in the development of services and features. Table of content - Datalake concept Scientific paper - The project : context - The project : design Architecture : Zone-based service-oriented datalake + Raw data management zone (a.k.a. landing area) + Metadata management zone + Process zone - REST API, interaction with code - Web GUI, interaction with interface + Processed data zone (a.k.a. consumption zone, access zone or gold zone) + Services zone + Security, authentication and monitoring zone Data lifecycle + Main data life cycle : Batch data - Light process - Heavy process + Extended data life cyc","default_branch":null,"files":null,"tree":[],"storefront":"/r/vincentnam","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/vincentnam/docker_datalake/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}