{"repo":"llphxd/spark-redis-connector","free":true,"listed":false,"github":"https://github.com/llphxd/spark-redis-connector","clone":"git clone https://github.com/llphxd/spark-redis-connector.git","description":"A modern Spark SQL DataSource V2 connector for Redis, focused on DataFrame and SQL support for Redis native data types.","language":"Scala","stars":26,"topics":["data-engineering","datasource-v2","redis","scala","spark","spark-sql"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"spark-redis-connector Language / 语言 English · 📖 中文文档 spark-redis-connector is a Spark SQL connector for Redis built around Spark DataSource V2. The project intentionally focuses on Spark SQL and DataFrame APIs only. It does not carry the legacy RDD, DStream, Spark Streaming, or DataSource V1 APIs from older Spark Redis connectors. Features - Spark SQL USING redis and DataFrame .format(\"redis\") support. - Batch read/write support for Redis string, hash, set, list, and sorted set. - Redis native data types are exposed as table layouts instead of legacy RDD APIs. - Redis key discovery uses SCAN for pattern-based reads. - A shaded all jar is published for local Spark testing. Redis Streams are intentionally left for a separate design because batch stream reads and structured streaming have different offset and commit semantics. Compatibility Component Supported version --- --- Spark 3.3.x , 3.4.x , 3.5.x Scala binary version 2.12 Java 8+ Redis client Jedis 5.1.5 Artifact spark-redis-connector 2.12 Spark 3.2.x and earlier, and Spark 4.x and later, are not declared as supported by this release. They may work, but you should validate them with your own Spark and Redis runtime tests. Quick Start Download a release jar from GitHub Releases. Each release provides two artifacts: - spark-redis-connector 2.12- -all.jar — connector plus shaded runtime dependencies ( recommended for Spark --jars ) - spark-redis-connector 2.12- .jar — connector classes only Replace with the release version ","default_branch":null,"files":null,"tree":[],"storefront":"/r/llphxd","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/llphxd/spark-redis-connector/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}