Search engine scraping: Difference between revisions

Revision as of 13:04, 27 July 2022

Revision as of 13:05, 27 July 2022

Line 62:

* [[cURL]] – a command line browser for automation and testing as well as a powerful open source HTTP interaction library available for a large range of programming languages.<ref>{{cite web|url=https://curl.haxx.se/libcurl/|title=libcurl - the multiprotocol file transfer library|website=curl.haxx.se}}</ref>

* google-search - A Go package to scrape Google. <ref>{{cite web|url=https://github.com/rocketlaunchr/google-search|title=A Go package to scrape Google.|via=GitHub}}</ref>

* [https://seotoolskit.co/ SEO Tools Kit] – Free Online Tools, Duckduckgo, Baidu, [[Petal Search|Petal]], [[Sogou]]) by using proxies (socks4/5, http proxy). The [https://seofixing.blogspot.com/ SEO Fixing] tool includes asynchronous networking support and is able to control real browsers to mitigate detection.<ref>{{cite web|url=https://seotoolskit.co/|title=Free online SEO Tools (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.: NikolaiT/SEO Tools Kit|date=15 January 2019|publisher=|via=GitHub}}</ref>

*se-scraper - Successor of SEO Tools Kit. Scrape search engines concurrently with different proxies. <ref>{{Citation|last=Tschacher|first=Nikolai|title=NikolaiT/se-scraper|date=2020-11-17|url=https://github.com/NikolaiT/se-scraper|access-date=2020-11-19}}</ref>