{"repo":"microlinkhq/metascraper","free":true,"listed":false,"github":"https://github.com/microlinkhq/metascraper","clone":"git clone https://github.com/microlinkhq/metascraper.git","description":"Scrape metadata from any URL using Open Graph, JSON-LD, HTML meta tags, and smart fallbacks.","language":"HTML","stars":2729,"topics":["metadata","article","cheerio","extract","html","json-ld","metascraper","og","open-graph","preview"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"A library to easily extract unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more. - What is it - Getting started - What data it detects - The cloud API solution - How it works - Importing rules - Rules bundles - Official - Community - API - constructor(rules) - rules - metascraper(options) - options - html - htmlDom - omitPropNames - pickPropNames - rules - url - validateUrl - Environment Variables - METASCRAPER\\ RE2 - Benchmark - License --- What is it The metascraper library allows you to easily scrape metadata from an article on the web using Open Graph metadata, regular HTML metadata, and a series of fallbacks. It follows a few principles: - Ensure a high accuracy for online articles by default. - Make it simple to add new rules or override existing ones. - Don't restrict rules to CSS selectors or text accessors. Getting started Below is a real example of extracting metadata from a live website. The same logic shown here is running online and can be tested directly at microlink.io/meta: metascraper requires two inputs: The target URL and the HTML markup behind that URL. There are multiple ways to retrieve the HTML markup, but it needs to be as accurate as possible. For that reason, we developed html-get, which uses a headless browser to retrieve HTML in a way that works seamlessly with metascraper. The output will be something like: What data it detects Note : Custom metadata detection can be defined using a rule bundle.","default_branch":null,"files":null,"tree":[],"storefront":"/r/microlinkhq","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/microlinkhq/metascraper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}