A good place to start finding answers is the most recent State of the Bots report from AI startup TollBit. For publishers that are feeling the heat of AI, it attaches real numbers to the presence of AI in the media ecosystem and how quickly it's growing. And while the rise of AI bots is a worrisome trend to those in the content business, it may also be an opportunity.
Google has one crawler, which means they use the same crawler for their search, where they still send us traffic, as they do for their AI products, where they steal our content,
The idea behind RSL is brutally simple. Instead of the old file -- which only said, "yes, you can crawl me," or "no, you can't," and which AI companies often ignore -- publishers can now add something new: machine-readable licensing terms. Want an attribution? You can demand it. Want payment every time an AI crawler ingests your work, or even every time it spits out an answer powered by your article?
Google's CEO, Sundar Pichai, said in May that web publishing is not dying. Nick Fox, VP of Search at Google, said in May that the web is thriving. But in a court document filed by Google on late Friday, Google's lawyers wrote, "The fact is that today, the open web is already in rapid decline."
Publishers are picking up new vibes from Meta, which they believe signal that the platform may be changing its stance on AI licensing. So far, it's more rhetoric than reality. Nevertheless, if it were to come to fruition, it could reset the dynamic between Meta and publishers, many of whom still feel burned by years of declining referral traffic from the platform.
Last year, Facebook moved away from a revenue-share payout model that paid publishers based on the performance of ads in content, transitioning to a model that rewards creators based on content performance itself. This shift has effectively changed how publishers engage with the platform, focusing more on the content they create for Facebook.