{"repo":"jdinkla/location-based-nearest-neighbours","free":true,"listed":false,"github":"https://github.com/jdinkla/location-based-nearest-neighbours","clone":"git clone https://github.com/jdinkla/location-based-nearest-neighbours.git","description":"Using k-d trees with Apache Spark and Scala","language":"Scala","stars":11,"topics":["apache-spark","big-data","kd-tree","nearest-neighbors","scala"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"lbnn - Location based nearest neighbors ======================================= This project shows how to use kd-trees with Apache Spark and Scala for the analysis of location based / geometric data. The example data consists of records of \"check ins\" of users with a mobile phone. For each checkin the user id, the date and time and the location coordinates (latitude and longitude) are recorded. The example data set is called \"loc-Gowalla\" and available at the Stanford Large Network Dataset Collection. With this app we can calculate for every user at some specified point in time the number of users that are checked in in his/her \"neighborhood\" (specified in kilometers). Tools used: Apache Hadoop, Apache Spark, Scala, SBT, ScalaTest, Kryo. Requirements: A JDK 17 — Spark 3.5 does not run on newer JDKs (e.g. Java 21/25). SBT 1.x — the pinned version is fetched automatically from project/build.properties . No Spark cluster is needed: the app runs locally out of the box ( spark.master=local[ ] in local.properties ). How to use ---------- Clone the repository and change into it: Run locally This needs only a JDK 17 and SBT — no Spark cluster. The repo ships a justfile with the common commands ( just --list to see them all); the equivalent raw SBT call is shown in the comment next to each. Download the example data, sort it by user, and find the neighbors for the 6th of October 2009 within a 5 km range for every user: Result and intermediate files are written under temp/ (configurabl","default_branch":null,"files":null,"tree":[],"storefront":"/r/jdinkla","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jdinkla/location-based-nearest-neighbours/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}