{"repo":"ml4t/data","free":true,"listed":false,"github":"https://github.com/ml4t/data","clone":"git clone https://github.com/ml4t/data.git","description":"Market data acquisition, storage, and update workflows for machine learning for trading.","language":"Python","stars":42,"topics":["algorithmic-trading","data-engineering","finance","financial-data","machine-learning-for-trading","market-data","ml4t","parquet","polars","python"],"license":"MIT","category":"trading","readme_excerpt":"ml4t-data Unified market data acquisition and storage for quantitative research workflows. Part of the ML4T Library Ecosystem This library is one of six interconnected libraries supporting the machine learning for trading workflow described in Machine Learning for Trading: Together they cover data infrastructure, feature engineering, modeling, signal evaluation, strategy backtesting, and live deployment. What This Library Does Quantitative research requires consistent, reproducible access to market data from multiple sources. ml4t-data provides: - DataManager as the unified OHLCV interface for registry providers with that capability - 20+ provider adapters covering equities, crypto, futures, forex, macro, prediction markets, and factors - Automated storage in Hive-partitioned Parquet format with metadata tracking - Incremental updates, gap detection, and backfill via CLI - Built-in data validation (OHLC invariants, deduplication, anomaly detection) - Futures module for CME/ICE bulk downloads with continuous contract construction - COT module for CFTC Commitment of Traders weekly reports - Resilience: rate limiting, retry with exponential backoff, gap detection The goal is to support an ongoing research workflow rather than one-off downloads. Data is stored locally, tracked for freshness, and queryable with tools like DuckDB or Polars. Installation ML4T Data supports stable CPython 3.12 through 3.14 on Linux, macOS, and Windows. Python 3.15 is not supported until the core depe","default_branch":null,"files":null,"tree":[],"storefront":"/r/ml4t","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ml4t/data/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}