{"repo":"analysiscenter/batchflow","free":true,"listed":false,"github":"https://github.com/analysiscenter/batchflow","clone":"git clone https://github.com/analysiscenter/batchflow.git","description":"BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.","language":"Python","stars":205,"topics":["python3","data-science","machine-learning","python","pipeline-framework","pipeline","workflow","workflow-engine"],"license":"Apache-2.0","category":"workflow-automation","readme_excerpt":"BatchFlow BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory. For more details see the documentation and tutorials. Main features: - flexible batch generaton - deterministic and stochastic pipelines - datasets and pipelines joins and merges - data processing actions - flexible model configuration - within batch parallelism - batch prefetching - ready to use ML models and proven NN architectures - convenient layers and helper functions to build custom models - a powerful research engine with parallel model training and extended experiment logging. Basic usage The trick here is that all the processing actions are lazy. They are not executed until their results are needed, e.g. when you request a preprocessed batch: or or Train a neural network BatchFlow includes ready-to-use proven architectures like VGG, Inception, ResNet and many others. To apply them to your data just choose a model, specify the inputs (like the number of classes or images shape) and call train model . Of course, you can also choose a loss function, an optimizer and many other parameters, if you want. For more advanced cases and detailed API see the documentation. Installation BatchFlow module is in the beta stage. Your suggestions and improvements are very welcome. BatchFlow supports Python 3.9 or higher. Stable python package With uv With poetry With old-fashioned pip De","default_branch":null,"files":null,"tree":[],"storefront":"/r/analysiscenter","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/analysiscenter/batchflow/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}