{"repo":"itinycheng/flink-platform-backend","free":true,"listed":false,"github":"https://github.com/itinycheng/flink-platform-backend","clone":"git clone https://github.com/itinycheng/flink-platform-backend.git","description":"Define and schedule workflow, support Flink Jar/SQL, ClickHouse/Hive/Mysql SQL, Shell, etc.","language":"Java","stars":22,"topics":["flink","workflow","airflow","azkaban","clickhouse","dolphinscheduler","oozie"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"flink-platform-web This project is a job scheduling framework with center less structure, easy to scale up. We can customize the workflow DAG and schedule it. preview video Overview 1. Quick Start 2. Configuration Details 3. Architecture Design - WebUI: Frontend files are in a separate project written in vue, please visit flink-platform-frontend. - Platform Instance: The instance for manage, configuration and scheduling workflow, easy to scale. - HDFS: Used to store resource files, such as jar, udf, etc. - Mysql: Holds all info about jobs, users, resources, schedules, etc. To keep the system simple, I plan to use mysql to guarantee fault-tolerance. Task Support - Flink sql/jar, deployment mode: YARN-Per-Job(tested), Other(untested). - Shell (tested). - SQL(ClickHouse/Mysql/Hive/TiDB tested). - Condition(tested, support: AND, OR). - Dependent(tested). - More: I don't have enough time to develop multi type task support, but implementing a new task type is easy, sometimes you can do it yourself or tell me your needs. Metadata Table Name Description :--------------- :--------------------------------------------------------------- t alert Alert configuration t catalog info Flink catalog configuration for FlinkSQL, modify in the future t job Job info t job flow Job flow info, workflow definition t job flow run Job flow execution instance t job run Job execution instance t resource Resources info, such as: jar, file, etc. t user login user info t user session login user session info","default_branch":null,"files":null,"tree":[],"storefront":"/r/itinycheng","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/itinycheng/flink-platform-backend/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}