{"repo":"nsoft/jesterj","free":true,"listed":false,"github":"https://github.com/nsoft/jesterj","clone":"git clone https://github.com/nsoft/jesterj.git","description":"Document Ingestion Framework for Search Systems","language":"Java","stars":37,"topics":["java","solr","elasticsearch","search"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Document Ingestion for Search A highly flexible, scalable, fault-tolerant document ingestion system designed for search. Builds are run on infrastructure kindly donated by   The problem Frequently, search projects start by feeding a few documents manually to a search engine, often via the \"just for testing\" built-in processing features of Solr such as SolrCell or post.jar. These features are documented and included to help users get a feel for what they can do with Solr with minimal painful setup. This is good, and that's how it should be for first explorations. Unfortunately, it's also a potential trap. Large-scale ingestion of documents for search is non-trivial. Many projects outgrow these simple tools and have to throw away their early exploratory work. Nobody likes setting aside valuable work, and it's natural to resist, but the longer one clings to an insufficient tool, the bigger, more difficult, and more expensive the migration is. Common problems are: - It works \"ok\" for a small test corpus and then becomes unstable on a larger production corpus. - The code written to feed into such interfaces (hopefully) reproduces standard solutions to problems that have been solved many times by other search engineers over the last 20 years. - No way to recover if indexing errors or is disrupted partway through. One is forced to start again from the beginning. - If failure is related to the size of a growing corpus, failures become increasingly common, and eventually, the sea","default_branch":null,"files":null,"tree":[],"storefront":"/r/nsoft","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/nsoft/jesterj/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}