{"repo":"UrbanOS-Public/smartcitiesdata","free":true,"listed":false,"github":"https://github.com/UrbanOS-Public/smartcitiesdata","clone":"git clone https://github.com/UrbanOS-Public/smartcitiesdata.git","description":"The core micro services of UrbanOS as an umbrella project with component documentation","language":"Elixir","stars":69,"topics":["elixir","elixir-phoenix","data-processing","data-analytics","data-visualization"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"Smart Cities Data Platform Project Description The platform is a combination of Elixir micro services custom built to ingest, normalize, transform, persist, and stream data from numerous sources, orchestrated via Kubernetes in any cloud provider or on-prem Kubernetes deployment. The loosely coupled services pass data across the pipeline via Kafka message queues and persist data to any hyper-scalable object store providing the S3 standard. They coordinate and communicate via a single event bus, also running on top of Kafka. The distributed data files are persisted and retrieved via SQL queries processed by the PrestoDB engine. Finally, user access, discovery, and analysis is facilitated by a ReactJS web application user interface, a RESTful API, or a web socket API for streaming data feeds. Microservices Application Short Description Build Status ----------------- ----------------- ------------ Andi Admin Interface for creating/editing datasets to be ingested Discovery API API to search for and query datasets Discovery Streams Websocket connection to listen to streaming data Estuary Microservice to persist event stream events Forklift Microservice for saving data to Presto DB Reaper Microservice to retrieve data Valkyrie Microservice to validate data structure during ingestion Alchemist Microservice to alter data from its original format Prerequisites General Prerequisites Elixir - The primary language that all of the microservices are written in Docker - All microservices are","default_branch":null,"files":null,"tree":[],"storefront":"/r/UrbanOS-Public","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/UrbanOS-Public/smartcitiesdata/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}