{"repo":"tweag/sparkle","free":true,"listed":false,"github":"https://github.com/tweag/sparkle","clone":"git clone https://github.com/tweag/sparkle.git","description":"Haskell on Apache Spark.","language":"Haskell","stars":449,"topics":["apache-spark","spark","haskell","analytics"],"license":"BSD-3-Clause","category":"analytics","readme_excerpt":"sparkle: Apache Spark applications in Haskell sparkle [spär′kəl]: a library for writing resilient analytics applications in Haskell that scale to thousands of nodes, using [Spark][spark] and the rest of the Apache ecosystem under the hood. See [this blog post][hello-sparkle] for the details. [spark]: http://spark.apache.org/ [hello-sparkle]: http://www.tweag.io/posts/2016-02-25-hello-sparkle.html Getting started The tl;dr using the hello app as an example on your local machine: You'll need [Nix][nix] for the above to work. How it works sparkle is a tool for creating self-contained Spark applications in Haskell. Spark applications are typically distributed as JAR files, so that's what sparkle creates. We embed Haskell native object code as compiled by GHC in these JAR files, along with any shared library required by this object code to run. Spark dynamically loads this object code into its address space at runtime and interacts with it via the Java Native Interface (JNI). How to use To run a Spark application the process is as follows: 1. create an application in the apps/ folder, in-repo or as a submodule; 1. build the app; 1. submit it to a local or cluster deployment of Spark. If you run into issues, read the Troubleshooting section below first. Build Linux Include the following in a BUILD.bazel file next to your source code. You might want to add the following settings to your .bazelrc.local file. And then ask [Bazel][bazel] to build a deploy jar file. Other platforms spar","default_branch":null,"files":null,"tree":[],"storefront":"/r/tweag","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/tweag/sparkle/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}