{"repo":"NailShakurov/logzip","free":true,"listed":false,"github":"https://github.com/NailShakurov/logzip","clone":"git clone https://github.com/NailShakurov/logzip.git","description":"Compress logs for LLM analysis — Rust-powered, Python API. 40-60% token savings.","language":"Rust","stars":12,"topics":["fastapi-docker-systemd-devtools-cli","log-compression-llm-rust-python-logging","log","logging","logs","cli","devtools","docker","fastapi","llm"],"license":"MIT","category":"ai-agents","readme_excerpt":"logzip (Rust) Compress logs before sending to LLM. Powered by Rust & PyO3. Before / After Raw Log (Uvicorn): logzip output: Typical savings: 52–58% on structured logs (systemd, uvicorn, docker). Anomalies and unique lines stay uncompressed — visible at a glance in the BODY. Compression is lossy-semantic by default (sub-second timestamps trimmed, whitespace collapsed — meaning preserved). Use --lossless for a byte-exact roundtrip. Why use logzip? (RAG & LLM) When working with logs in LLMs (Claude, GPT, RAG systems), you face two problems: 1. Context Limit : Logs are huge. A 10MB log is 2.5M tokens. 2. Noise : 90% of the log consists of repeating INFO and identical requests that drown out the real error. logzip is well-suited for RAG pipelines : it compresses the context before sending it to the model, saving money on tokens and increasing answer accuracy by highlighting anomalies. --- Performance (7.96 MB Log, 2M tokens) Benchmarked on a real 7.96 MB production log. logzip modes Mode CLI Time (ms) Size (KB) Saved (%) Output type :--- :--- :--- :--- :--- :--- fast --quality fast 200 4,900 40% text/LLM balanced --quality balanced 404 3,928 52% text/LLM balanced + 2 passes ★ --quality balanced --bpe-passes 2 418 3,404 58% text/LLM max --quality max 1,600 ≤ 3,404 ≥ 58% text/LLM ★ Recommended default. A second compression pass finds repeated token sequences in already-compressed text — 14 ms overhead, 7% more savings vs balanced . --quality max runs an auto-search over several (leg","default_branch":null,"files":null,"tree":[],"storefront":"/r/NailShakurov","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/NailShakurov/logzip/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}