{"repo":"jae-jae/QueryList","free":true,"listed":false,"github":"https://github.com/jae-jae/QueryList","clone":"git clone https://github.com/jae-jae/QueryList.git","description":":spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。","language":"PHP","stars":2692,"topics":["querylist","crawler","spider","scraper"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"QueryList QueryList is a simple, elegant, extensible PHP Web Scraper (crawler/spider) ,based on phpQuery. API Documentation 中文文档 Sponsors QueryList users can get 3 GB of proxy traffic for testing with promo code QueryList3GB from SX.org. Reliable HTTP and SOCKS5 proxies for web scraping and data collection SX.org . Features - Have the same CSS3 DOM selector as jQuery - Have the same DOM manipulation API as jQuery - Have a generic list crawling program - Have a strong HTTP request suite, easy to achieve such as: simulated landing, forged browser, HTTP proxy and other complex network requests - Have a messy code solution - Have powerful content filtering, you can use the jQuey selector to filter content - Has a high degree of modular design, scalability and strong - Have an expressive API - Has a wealth of plug-ins Through plug-ins you can easily implement things like: - Multithreaded crawl - Crawl JavaScript dynamic rendering page (PhantomJS/headless WebKit) - Image downloads to local - Simulate browser behavior such as submitting Form forms - Web crawler - ..... Requirements - PHP = 8.1 Installation By Composer installation: Usage DOM Traversal and Manipulation - Crawl「GitHub」all picture links - Crawl Google search results - More usage List crawl Crawl the title and link of the Google search results list: Results: Encode convert HTTP Client (GuzzleHttp) - Carry cookie login GitHub - Use the Http proxy For proxy setup, you can also consider SX.org. - Analog login Submit forms ","default_branch":null,"files":null,"tree":[],"storefront":"/r/jae-jae","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jae-jae/QueryList/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}