{"repo":"skrapeit/skrape.it","free":true,"listed":false,"github":"https://github.com/skrapeit/skrape.it","clone":"git clone https://github.com/skrapeit/skrape.it.git","description":"A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.","language":"Kotlin","stars":875,"topics":["skrape","dom","parse","kotlin","html-parser","test-automation","crawler","scraper","testing","kotlin-dsl"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"skrape{it} ==================================== :bellhop bell: :rotating light: Help wanted :rotating light: :bellhop bell: ------------- Looking for Co-Maintainer(s), please contact christian.draeger1@gmail.com if you are interested in helping to maintain and evolve skrape{it} :heart: skrape{it} is a Kotlin-based HTML/XML testing and web scraping library that can be used seamlessly in Spring-Boot, Ktor, Android or other Kotlin-JVM projects. The ability to analyze and extract HTML including client-side rendered DOM trees and all other XML-related markup specifications such as SVG, UML, RSS,... makes it unique. It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. First and foremost skrape{it} aims to be a testing tool (not tied to a particular test runner), but it can also be used to scrape websites in a convenient fashion. Features Parsing - [x] Deserialization of HTML/XML from websites, local html files and html as string to data classes / POJOs. - [x] Designed to deserialize HTML but can handle any XML-related markup specifications such as SVG, UML, RSS or XML itself. - [x] DSL to select html elements as well as supporting CSS query-selector syntax by string invocation. Http-Client - [x] Http-Client without verbosity and ceremony to make requests and corresponding request options like headers, cookies etc. in a fluent style interface. - [x] Pre-configure client regarding auth and other request settings - [x] Can handle ","default_branch":null,"files":null,"tree":[],"storefront":"/r/skrapeit","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/skrapeit/skrape.it/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}