{"repo":"caiopizzol/fipe-data-pipeline","free":true,"listed":false,"github":"https://github.com/caiopizzol/fipe-data-pipeline","clone":"git clone https://github.com/caiopizzol/fipe-data-pipeline.git","description":"Coleta e processa dados históricos de preços da Tabela FIPE para PostgreSQL.","language":"TypeScript","stars":50,"topics":["fipe","fipe-api","bun","data-pipeline","postgresql","tabela-fipe","typescript","web-scraper"],"license":null,"category":"data-pipelines","readme_excerpt":"FIPE Data Pipeline Coleta e processa dados históricos de preços de veículos da Tabela FIPE para PostgreSQL. Parte do fipe.chat — os dados que a FIPE tem, a clareza que ela nunca deu. Requisitos - Bun — brew install oven-sh/bun/bun - Docker Início Rápido Comandos Uso Monthly refresh The public site reads latest prices , which only includes reference tables with reference tables.published at IS NOT NULL . Use the refresh command for the monthly publication flow: refresh takes a session-level Postgres advisory lock named fipe refresh . If another refresh is already running, it logs that state and exits 0 , which is expected when a long crawl overlaps the next cron tick. It publishes only official reference tables newer than the latest published month, oldest first. Each candidate is crawled, then published only when checkpoint backlog is zero and its price count is at least 90% of the previous published reference's price count. Failed crawl or validation exits 1 without setting published at , so the next run resumes from checkpoints. If a materialized-view refresh fails after published at is set, a later refresh run retries latest prices before reporting success. Set HC REFRESH URL to enable best-effort healthchecks: /start after the advisory lock is acquired, the base URL on success, and /fail on failure. Lock-held no-op ticks ping nothing, so a wedged multi-day run shows up as a missed ping instead of being masked by daily \"already running\" successes; configure the check with ","default_branch":null,"files":null,"tree":[],"storefront":"/r/caiopizzol","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/caiopizzol/fipe-data-pipeline/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}