{"repo":"mujib77/pgstream","free":true,"listed":false,"github":"https://github.com/mujib77/pgstream","clone":"git clone https://github.com/mujib77/pgstream.git","description":"Real-time PostgreSQL WAL reader built in GO - captures INSERT, UPDATE, DELETE using logical replication","language":"Go","stars":35,"topics":["cdc","database-internals","golang","logical-replication","mermaidjs","postgresql","wal"],"license":"MIT","category":"databases-storage","readme_excerpt":"pgstream A real-time PostgreSQL WAL reader built in Go. Captures every INSERT, UPDATE, and DELETE from your Postgres database as it happens. What is pgstream? Every time you insert, update, or delete a row in PostgreSQL, Postgres writes that change to a file called the WAL (Write Ahead Log) before touching the actual data. This is how Postgres makes sure nothing gets lost if the server crashes. pgstream taps into that log and streams every change out in real time. You can see exactly what changed, in which table, and what the old and new values were. This is the same mechanism that powers production CDC (Change Data Capture) tools like Debezium and pglogical used by companies running Postgres at scale. How it works Replication Slot pgstream creates a replication slot in your Postgres database. Think of it as a bookmark in the WAL stream. Postgres tracks your position and makes sure no changes get deleted until pgstream has read them. pgoutput WAL records are stored in raw binary. pgoutput is a built-in Postgres plugin that decodes that binary into readable messages like INSERT, UPDATE, DELETE along with the actual row data. LSN (Log Sequence Number) Every record in the WAL has a unique position called an LSN. pgstream tracks the LSN of every event so you always know exactly where in the stream each change came from. Architecture Project Structure Setup Prerequisites: - PostgreSQL with logical replication enabled - Go 1.21 or higher 1. Enable logical replication In postgresql.","default_branch":null,"files":null,"tree":[],"storefront":"/r/mujib77","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/mujib77/pgstream/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}