Anubis is a software program that adds a JavaScript-powered proof of work challenge to websites before users can access them, as a form of deterring the impacts of web scraping from bots. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]
Anubis | |
---|---|
![]() The page shown by Anubis during its challenge. | |
Original author(s) | Xe Iaso |
Developer(s) | Techaro[1] |
Initial release | January 19, 2025[1][2] |
Stable release | 1.21.3[3] ![]() |
Repository | github |
Written in | Go, JavaScript[1] |
License | MIT License |
Website | anubis |
It was created by Xe Iaso in response to Amazon's web crawler overloading their Git server, as it did not respect the robots.txt exclusion protocol and would work around restrictions.[4][6]
It is used by a number of projects, including:[7]
References
edit- ^ a b c d e Brockmeier, Joe (July 10, 2025). "Anubis sends AI scraperbots to a well-deserved fate". LWN.net.
- ^ Iaso, Xe (January 19, 2025). "Block AI scrapers with Anubis". xeiaso.net. Retrieved May 2, 2025.
- ^ "Release 1.21.3". July 25, 2025. Retrieved August 12, 2025.
- ^ a b c Edwards, Benj (March 25, 2025). "Open source devs say AI crawlers dominate traffic, forcing blocks on entire countries". Ars Technica. Retrieved May 2, 2025.
- ^ Bort, Julie (March 27, 2025). "Open source devs are fighting AI crawlers with cleverness and vengeance". TechCrunch. Retrieved May 2, 2025.
- ^ a b c Maiberg, Emanuel (July 7, 2025). "The Open-Source Software Saving the Internet From AI Bot Scrapers". 404 Media. Retrieved July 14, 2025.
- ^ a b "List of known websites using Anubis". anubis.techaro.lol. Retrieved May 10, 2025.
- ^ "Anubis". Techaro. Retrieved July 28, 2025.
- ^ a b c Iaso, Xe (April 12, 2025). "Anubis works". xeiaso.net. Retrieved May 9, 2025.
- ^ "Forum and Wiki Performance; Anubis Deployed". FreeCAD News. April 30, 2025. Retrieved May 11, 2025.
- ^ "The Day Anubis Saved Our Websites From a DDoS Attack". fabulous.systems. May 1, 2025. Retrieved June 19, 2025.