{"repo":"microsoft/data-accelerator","free":true,"listed":false,"github":"https://github.com/microsoft/data-accelerator","clone":"git clone https://github.com/microsoft/data-accelerator.git","description":"Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.","language":"C#","stars":312,"topics":["spark","spark-streaming","spark-sql","sparksql","streaming-data","streaming","servicefabric","nodejs","docker","hdinsight"],"license":"MIT","category":"deployment-docker-iac","readme_excerpt":"Data Accelerator for Apache Spark Flow Gateway DataProcessing :---: :-----: :-----: :-----: :-----: :-----: Metrics SimulatedData Website Data Accelerator for Apache Spark democratizes streaming big data using Spark by offering several key features such as a no-code experience to set up a data pipeline as well as fast dev-test loop for creating complex logic. Our team has been using the project for two years within Microsoft for processing streamed data across many internal deployments handling data volumes at Microsoft scale. It offers an easy to use platform to learn and evaluate streaming needs and requirements. We are thrilled to share this project with the wider community as open source! Azure Friday: We are now featured on Azure Fridays! See the video here. Data Accelerator offers three level of experiences: - The first requires no code at all, using rules to create alerts on data content. - The second allows to quickly write a Spark SQL query with additions like LiveQuery, time windowing, in-memory accumulator and more. - The third enables integrating custom code written in Scala or via Azure functions. You can get started locally for Windows, macOs and Linux following these instructions To deploy to Azure, you can use the ARM template; see instructions deploy to Azure. The data-accelerator repository contains everything needed to set up an end-to-end data pipeline. There are many ways you can participate in the project: - Submit bugs and requests - Review code changes","default_branch":null,"files":null,"tree":[],"storefront":"/r/microsoft","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/microsoft/data-accelerator/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}