{"repo":"andrewm4894/airflow-provider-anomaly-detection","free":true,"listed":false,"github":"https://github.com/andrewm4894/airflow-provider-anomaly-detection","clone":"git clone https://github.com/andrewm4894/airflow-provider-anomaly-detection.git","description":"An airflow provider for anomaly detection.","language":"Python","stars":22,"topics":["airflow","airflow-providers","anomaly-detection","machine-learning","pyod"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Anomaly Detection with Apache Airflow Painless anomaly detection (using PyOD) with Apache Airflow. Note: If you are interested in a more generic approach to anomaly detection that does not need or use Airflow, check out Anomstack - How - Example Alert - Alert Text (ascii art yay!) - Alert Chart - Getting Started - Prerequisites - Installation - Configuration - Docker - Anomaly Gallery How How it works in a 🌰: 1. Create and express your metrics via SQL queries (example here). 1. Some YAML configuration fun (example here, defaults here). 1. Receive useful alerts when metrics look anomalous (example here). The example dag will create 4 dags for each \"metric batch\" (a metric batch is just the resulting table of 1 or more metrics created in step 1 above): - ingestion : Ingests the metric data into a table in BigQuery. - training : Uses recent metrics and preprocess.sql to train an anomaly detection model for each metric and save it to GCS. - scoring : Uses latest metrics and preprocess.sql to score recent data using latest trained model. - alerting : Uses recent scores and alert status.sql to trigger an alert email if alert conditions are met. Example Alert Example output of an alert. Horizontal bar chart used to show metric values over time. Smoothed anomaly score is shown as a % and any flagged anomalies are marked with . In the example below you can see that the anomaly score is elevated when the metric dips and also when it spikes. Alert Text (ascii art yay!) Below is the sql","default_branch":null,"files":null,"tree":[],"storefront":"/r/andrewm4894","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/andrewm4894/airflow-provider-anomaly-detection/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}