{"repo":"jncraton/languagemodels","free":true,"listed":false,"github":"https://github.com/jncraton/languagemodels","clone":"git clone https://github.com/jncraton/languagemodels.git","description":"Explore large language models in 512MB of RAM","language":"HTML","stars":1193,"topics":["llm","nlp","python"],"license":"MIT","category":"machine-learning","readme_excerpt":"Language Models =============== Python building blocks to explore large language models in as little as 512MB of RAM This package makes using large language models from Python as simple as possible. All inference is performed locally to keep your data private by default. Installation and Getting Started -------------------------------- This package can be installed using the following command: Once installed, you should be able to interact with the package in Python as follows: This will require downloading a significant amount of data ( 250MB) on the first run. Models will be cached for later use and subsequent calls should be quick. Example Usage ------------- Here are some usage examples as Python REPL sessions. This should work in the REPL, notebooks, or in traditional scripts and applications. Instruction Following Outputs can be restricted to a list of choices if desired: Adjusting Model Performance The base model should run quickly on any system with 512MB of memory, but this memory limit can be increased to select more powerful models that will consume more resources. Here's an example: GPU Acceleration If you have an NVIDIA GPU with CUDA available, you can opt in to using the GPU for inference: Text Completions External Retrieval Helper functions are provided to retrieve text from external sources that can be used to augment prompt context. Here's an example showing how this can be used (compare to previous chat example): Semantic Search Semantic search is provided t","default_branch":null,"files":null,"tree":[],"storefront":"/r/jncraton","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jncraton/languagemodels/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}