{"repo":"LucaCanali/Miscellaneous","free":true,"listed":false,"github":"https://github.com/LucaCanali/Miscellaneous","clone":"git clone https://github.com/LucaCanali/Miscellaneous.git","description":"Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing, measuring CPUs' performance, and I/O latency heat maps. Jupyter notebooks examples for using various DB systems.","language":"Jupyter Notebook","stars":464,"topics":["apache-spark","database","jupyter-notebooks","performance-analysis","performance-monitoring","performance-testing"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"Miscellaneous Projects, Tools, and Scripts Contact: Luca.Canali@cern.ch Performance Engineering and Apache Spark Folder Description ------------------------------------------------------------------ ------------------------------------------------------------------------------------- Spark Dashboard A tool for Apache monitoring, use to build a performance dashboard and troubleshoot Spark jobs. Spark Notes Miscellaneous tips and code snippets about Apache Spark. Spark for Physics Examples, with code and data of using Apache Spark for High Energy Physics data analysis. Performance Testing Includes: - TPCDS-PySpark, run TPCDS bemchmark at scale with PySpark and collect execution metrics - Load testing tools for CPU benchmarking, in Python and Rust - Notes on how to use various tools for performance investigations Data Engineering and Data Science Folder Description ----------------------------------------------------------------------- ------------------------------------------------------------------------------------- Kepler Analysis A curated collection of interactive notebooks for executing Kepler's orbital analysis on Mars. Deep Learning Notes Notes and examples on Deep Learning tools and related data pipelines. Pyspark SQL Magic Jupyter How to write Jupyter SQL magic functions for PySpark and Spark SQL. Trino and Presto on Jupyter Example of using Trino or Presto on a Jupyter notebook. PostgreSQL and YugabyteDB on Jupyter Example of using PostgreSQL or YugabyteDB on a Jupy","default_branch":null,"files":null,"tree":[],"storefront":"/r/LucaCanali","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/LucaCanali/Miscellaneous/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}