{"repo":"alexeykarnachev/full_stack_transformer","free":true,"listed":false,"github":"https://github.com/alexeykarnachev/full_stack_transformer","clone":"git clone https://github.com/alexeykarnachev/full_stack_transformer.git","description":"Pytorch library for end-to-end transformer models training, inference and serving","language":"Python","stars":70,"topics":["transformers","pytorch-lightning","language-model","telegram-bot","language-model-generation","language-model-finetuning","gpt-2"],"license":null,"category":"machine-learning","readme_excerpt":"Full Stack Transformer Pytorch library for end-to-end transformer models training, inference and serving. Powered by a list of great libraries. Library Design The library is organized in such a way, that the core sub-package contains all modelling, data structures and data streaming classes. In tasks sub-package there are all tasks-specific code. Available tasks: - Document Language Model - the classic document-base language model training. It also provides application serving for interactive text generation. Document Language Model For now, there is only 1 task available in the library. So, I'll put an example of this task usage right here in README. When another tasks will be implemented, I'll move all examples in the documentation. Features - Automatic LM dataset preparation - End-to-end transformer LM training - Unlikelihood loss training - Training LM with meta data (control codes, CTRL) - Text generation tricks (top-k, nucleus, repetition penalty, etc) - Text generation as a service - Telegram bot client Prepare dataset First, you need two text files (train and validation) which contain raw documents. For this example, files are already placed here: (If you want to start with your own files, check Input Document Files Format Train Model The library uses pytorch-lightning for training and arguments which are used by lightning Trainer class are allowed as a command-line arguments for the script below. To check them all (as well as task-specific args) execute: Now, let's t","default_branch":null,"files":null,"tree":[],"storefront":"/r/alexeykarnachev","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/alexeykarnachev/full_stack_transformer/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}