{"repo":"explosion/spacy-llm","free":true,"listed":false,"github":"https://github.com/explosion/spacy-llm","clone":"git clone https://github.com/explosion/spacy-llm.git","description":"🦙 Integrating LLMs into structured NLP pipelines","language":"Python","stars":1393,"topics":["large-language-models","llm","openai","spacy","dolly","gpt-3","gpt-4","machine-learning","named-entity-recognition","natural-language-processing"],"license":"MIT","category":"ai-agents","readme_excerpt":"spacy-llm Structured NLP with LLMs This package integrates Large Language Models (LLMs) into spaCy, featuring a modular system for fast prototyping and prompting , and turning unstructured responses into robust outputs for various NLP tasks, no training data required. Feature Highlight - Serializable llm component to integrate prompts into your spaCy pipeline - Modular functions to define the task (prompting and parsing) and model - Interfaces with the APIs of - OpenAI - Cohere - Anthropic - Google PaLM - Microsoft Azure AI - Supports open-source LLMs hosted on Hugging Face 🤗: - Falcon - Dolly - Llama 2 - OpenLLaMA - StableLM - Mistral - Integration with LangChain 🦜️🔗 - all langchain models and features can be used in spacy-llm - Tasks available out of the box: - Named Entity Recognition - Text classification - Lemmatization - Relationship extraction - Sentiment analysis - Span categorization - Summarization - Entity linking - Translation - Raw prompt execution for maximum flexibility - Soon: - Semantic role labeling - Easy implementation of your own functions via spaCy's registry for custom prompting, parsing and model integrations. For an example, see here. - Map-reduce approach for splitting prompts too long for LLM's context window and fusing the results back together 🧠 Motivation Large Language Models (LLMs) feature powerful natural language understanding capabilities. With only a few (and sometimes no) examples, an LLM can be prompted to perform custom NLP tasks suc","default_branch":null,"files":null,"tree":[],"storefront":"/r/explosion","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/explosion/spacy-llm/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}