AI crawlers from companies like Amazon are overwhelming open source software services, such as Git repositories, causing significant downtime and instability. Developer Xe Iaso, faced with persistent bot traffic, created a custom proof-of-work challenge system named 'Anubis' to mitigate the issue. This situation reflects a broader crisis in the open source community, where reports indicate that up to 97% of some projects' traffic comes from AI bots, leading to increased operational costs and burdens on maintainers.
It's futile to block AI crawler bots because they lie, change their user agent, use residential IP addresses as proxies, and more.
Some open source projects now see as much as 97 percent of their traffic originating from AI companies' bots, dramatically increasing bandwidth costs.
Collection
[
|
...
]