{"repo":"mip404/jacana","free":true,"listed":false,"github":"https://github.com/mip404/jacana","clone":"git clone https://github.com/mip404/jacana.git","description":"A high-throughput Solana Geyser plugin for real-time blockchain data ingestion using ClickHouse as its columnar data store","language":"Rust","stars":18,"topics":["geyser-plugin","rust","solana","clickhouse"],"license":null,"category":"blockchain-web3","readme_excerpt":"Jacana A high-throughput Solana Geyser plugin for ClickHouse indexing using Apache Arrow columnar format over native TCP. Jacana eliminates the serialization overhead and row-wise conversion tax of traditional indexers by streaming Apache Arrow columnar batches directly to ClickHouse over the native binary protocol (port 9000). This architecture delivers sustained high throughput with predictable low latency. Architecture Design Pattern: Staged Event-Driven Architecture (SEDA) Jacana implements a multi-stage pipeline with isolated worker pools. Key Patterns: - Manager-Worker (Boss-Worker) : Each entity type has a manager coordinating multiple workers - Producer-Consumer : Bounded channels with backpressure handling - Bulkhead : Independent failure domains per entity type - Batching/Aggregation : Memory-efficient columnar batching before flush - Resource Pool : Shared connection pool prevents connection explosion High-Performance Channels: Jacana uses Kanal for inter-thread communication, providing: - Zero-allocation fast path : For small messages (≤ pointer size), data is encoded directly in the pointer - Stack-to-stack transfers : For larger messages, direct memory copying eliminates heap allocations - Bounded channels with backpressure : Configurable buffer sizes prevent memory exhaustion Why This Architecture Outperforms Traditional Indexers 1. Columnar Format End-to-End Traditional pipeline: Jacana pipeline: Benefits: - Zero intermediate serialization : No JSON encoding, ","default_branch":null,"files":null,"tree":[],"storefront":"/r/mip404","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/mip404/jacana/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}