{"repo":"benedekh/bigdata-projects","free":true,"listed":false,"github":"https://github.com/benedekh/bigdata-projects","clone":"git clone https://github.com/benedekh/bigdata-projects.git","description":"Student projects in Big Data field.","language":"Java","stars":18,"topics":["bigdata","big-data","apache-spark","hadoop","spark","hadoop-mapreduce","mapreduce"],"license":null,"category":"data-pipelines","readme_excerpt":"bigdata-projects Student projects in Big Data field, during my studies at Budapest University of Technology and Economics (BME). All the projects were homework assignments which were implemented by me. Comments and suggestions are warmly welcome. This repository uses large files. Please consider integrating Git LFS into your Git workflow as well! (Track .7z files.) Projects Flight data analysis Task Flight records in USA are stored and some of them are made available for research purposes at Statistical Computing. The data are separated by year from 1987 to 2008. The attributes include the common properties a flight record have (e.g. date, origin and destination airports, air time, scheduled and actual departure and arrival times, etc). During a practical course called 'Big Data Analytics Tools with Open-Source Platforms' at BME we had a homework assignment which contained two questions. The questions had to be answered by implementing a data analysis chain, that retrieves the neccesary information from the input files. We could use several technologies from the Hadoop Framework. I used Apache Spark™ and native Java MapReduce. We had to work with the dataset of year 2008, that has been stored on the datasets branch of this as well. Question #1 From which airport took most of the airplanes off? Answer #1 ATL, 408683 times According to the dataset of year 2008. I used Apache Spark™ and Java MapReduce to answer the question. The Apache Spark™ solution is avaiable here, while the","default_branch":null,"files":null,"tree":[],"storefront":"/r/benedekh","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/benedekh/bigdata-projects/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}