IPv6 zones in URLs are a mistake
4 hours ago
- #anti-scraping
- #proof-of-work
- #web security
- Anubis is a Proof-of-Work system used to protect the server from aggressive AI web scraping, which can cause downtime.
- It implements a scheme similar to Hashcash, making mass scraping more expensive while minimizing individual user impact.
- The tool is a temporary solution, with plans to improve fingerprinting techniques to identify headless browsers and reduce the need for challenge pages.
- Anubis requires modern JavaScript features, so disabling plugins like JShelter may be necessary for access.