{"repo":"dromara/dataCompare","free":true,"listed":false,"github":"https://github.com/dromara/dataCompare","clone":"git clone https://github.com/dromara/dataCompare.git","description":"big data comparison and data profiling platform: low code，data comparison and data profiling","language":"Java","stars":280,"topics":["bigdata","hive","spark","doris","low-code"],"license":"Apache-2.0","category":"workflow-automation","readme_excerpt":"dataCompare Introduction dataCompare is a database comparison and profiling platform (1)support Hive table data comparison, MySQL、Doris data comparison, realize automatic configuration for data comparison, avoid frequent SQL writing for processing (2)support easy configuration for data profiling Features data-compare (1)Interface-level interactive data comparison task configuration, low code and small amount of configuration to quickly generate comparison tasks (2)Magnitude comparison, consistency comparison, automatic difference case discovery (3)JDBC databases such as MySQL, Apache Hive, and Apache Doris are currently supported (4)Already supports the comparison results to automatically send email alarm reports data-profiling (1)Data detection can be completed with low code and a small amount of configuration (2)Primary key, enumeration value, null value detection Software Architecture Technology stack: End：Spring boot + Mybatis DataBase:MySQL Parsing Engine：Antrl Big Data：Hive、Spark System flowchart Demonstration of system functionality Home data-compare: DbConfig mysql config hive config job config Comparison results are displayed data-profiling: job config profiling result The system running environment java jdk8 mysql 5.7.36 Runing config (1)Run the SQL files in the SQL directory in the database, create database and tables (2)build jar using the source code of the project: mvn clean package (3)edit database config information of application.yml (4)run java -jar -Dspring","default_branch":null,"files":null,"tree":[],"storefront":"/r/dromara","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/dromara/dataCompare/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}