Search engine scraping: Difference between revisions

Revision as of 06:33, 28 January 2024

Revision as of 20:50, 10 February 2024

Line 28:

When search engine defense thinks an access might be automated, the search engine can react differently.

The first layer of defense is a captcha page<ref>{{Cite web|url=https://support.google.com/recaptcha/answer/6081888?hl=en|title=My computer is sending automated queries – reCAPTCHA Help|website=support.google.com|language=en|accessdate=2017-04-02}}</ref> where the user is prompted to verify they are a real person and not a bot or tool. Solving the [[CAPTCHA|captcha]] will create a [[HTTP cookie|cookie]] that permits access to the search engine again for a while. After about one day, the captcha page is removeddisplayed again.

The second layer of defense is a similar error page but without captcha, in such a case the user is completely blocked from using the search engine until the temporary block is lifted, or the user changes their IP.