{"repo":"dotnetcore/DotnetSpider","free":true,"listed":false,"github":"https://github.com/dotnetcore/DotnetSpider","clone":"git clone https://github.com/dotnetcore/DotnetSpider.git","description":"DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework","language":"C#","stars":4139,"topics":["crawler","dotnetcore","cross-platform","csharp","distributed"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"DotnetSpider 免责申明：本框架是为了帮助开发人员简化开发流程、提高开发效率，请勿使用此框架做任何违法国家法律的事情，使用者所做任何事情也与本框架的作者无关。 DotnetSpider, a .NET Standard web crawling library. It is a lightweight, efficient, and fast high-level web crawling & scraping framework. If you want to get the latest beta packages, you should add the myget feed: DESIGN DEVELOP ENVIROMENT 1. Visual Studio 2017 (15.3 or later) or Jetbrains Rider 2. .NET Core 2.2 or later 3. Docker 4. MySql docker run --name mysql -d -p 3306:3306 --restart always -e MYSQL ROOT PASSWORD=1qazZAQ! mysql:5.7 5. Redis (option) docker run --name redis -d -p 6379:6379 --restart always redis 6. SqlServer docker run --name sqlserver -d -p 1433:1433 --restart always -e 'ACCEPT EULA=Y' -e 'SA PASSWORD=1qazZAQ!' mcr.microsoft.com/mssql/server:2017-latest 8. PostgreSQL (option) docker run --name postgres -d -p 5432:5432 --restart always -e POSTGRES PASSWORD=1qazZAQ! postgres 9. MongoDb (option) docker run --name mongo -d -p 27017:27017 --restart always mongo 10. RabbitMQ docker run -d --restart always --name rabbimq -p 4369:4369 -p 5671-5672:5671-5672 -p 25672:25672 -p 15671-15672:15671-15672 \\ -e RABBITMQ DEFAULT USER=user -e RABBITMQ DEFAULT PASS=password \\ rabbitmq:3-management 11. Docker remote api for mac docker run -d --restart always --name socat -v /var/run/docker.sock:/var/run/docker.sock -p 2376:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock 12. HBase docker run -d --restart always --name hbase -p 20550:8080 -p 8085:8085 -p 9","default_branch":null,"files":null,"tree":[],"storefront":"/r/dotnetcore","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/dotnetcore/DotnetSpider/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}