{"repo":"AbsaOSS/cobrix","free":true,"listed":false,"github":"https://github.com/AbsaOSS/cobrix","clone":"git clone https://github.com/AbsaOSS/cobrix.git","description":"A COBOL parser and Mainframe/EBCDIC data source for Apache Spark","language":"Scala","stars":170,"topics":["cobol-parser","spark","cobol","copybook","mainframe","ebcdic","scalable","etl"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Cobrix - COBOL Data Source for Apache Spark Pain free Spark/Cobol files integration. Seamlessly query your COBOL/EBCDIC binary files as Spark Dataframes and streams. Add mainframe as a source to your data engineering strategy. Motivation Among the motivations for this project, it is possible to highlight: - Lack of expertise in the Cobol ecosystem, which makes it hard to integrate mainframes into data engineering strategies. - Lack of support from the open-source community to initiatives in this field. - The overwhelming majority (if not all) of tools to cope with this domain are proprietary. - Several institutions struggle daily to maintain their legacy mainframes, which prevents them from evolving to more modern approaches to data management. - Mainframe data can only take part in data science activities through very expensive investments. Features - Supports primitive types (although some are \"Cobol compiler specific\"). - Supports REDEFINES, OCCURS and DEPENDING ON fields (e.g. unchecked unions and variable-size arrays). - Supports nested structures and arrays. - Supports Hadoop (HDFS, S3, ...) as well as local file system. - The COBOL copybooks parser doesn't have a Spark dependency and can be reused for integrating into other data processing engines. - Supports reading files compressed in Hadoop-compatible way (gzip, bzip2, etc), but with limited parallelism. Uncompressed files are preferred for performance. Videos We have presented Cobrix at DataWorks Summit 2019 and Sp","default_branch":null,"files":null,"tree":[],"storefront":"/r/AbsaOSS","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/AbsaOSS/cobrix/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}