LLM scraper bots are overloading acme.com's HTTPS server
8 hours ago
- #network-outage
- #web-traffic
- #https-congestion
- Network outages at acme.com occurred intermittently from February 25th for over a month, with high ping times and packet drops lasting hours.
- Outages began after Sonic internet provider performed scheduled maintenance and switched to a new network.
- Collaboration with Sonic support did not resolve the issues despite identifying and fixing some configuration errors.
- The root cause was traced to the HTTPS server falling behind under increased traffic after the maintenance, causing congestion in the Network Address Translation daemon (natd).
- Temporarily closing port 443 (HTTPS) immediately resolved the outages, as HTTP traffic accounts for 90% of legitimate web traffic.
- The problem is widespread, affecting other hobbyist-level sites due to LLM companies scraping the web, highlighting a broader issue.