{"repo":"pemistahl/lingua-rs","free":true,"listed":false,"github":"https://github.com/pemistahl/lingua-rs","clone":"git clone https://github.com/pemistahl/lingua-rs.git","description":"The most accurate natural language detection library for Rust, suitable for short text and mixed-language text","language":"Rust","stars":1120,"topics":["rust","rust-library","rust-crate","language-detection","language-classification","language-recognition","natural-language-processing","nlp","language-identification","nlp-machine-learning"],"license":"Apache-2.0","category":"machine-learning","readme_excerpt":"1. What does this library do? Its task is simple: It tells you which language some text is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages. 2. Why does this library exist? Language detection is often done as part of large machine learning frameworks or natural language processing applications. In cases where you don't need the full-fledged functionality of those systems or don't want to learn the ropes of those, a small flexible library comes in handy. So far, other comprehensive open source libraries in the Rust ecosystem for this task are CLD2 , Whatlang and Whichlang . Unfortunately, most of them have two major drawbacks: 1. Detection only works with quite lengthy text fragments. For very short text snippets such as Twitter messages, they do not provide adequate results. 2. The more languages take part in the decision process, the less accurate are the detection results. Lingua aims at eliminating these problems. She nearly does not need any configuration and yields pretty accurate results on both long and short text, even on single words and phrases. She draws on both rule-based and statistical Naive Bayes methods but does not use neural networks or any dictionaries of words. She does not need a con","default_branch":null,"files":null,"tree":[],"storefront":"/r/pemistahl","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/pemistahl/lingua-rs/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}