There is tremendous concern in the media industry that AI platforms do not send traffic to publishers in return for scraping their content.
In Europe, independent publishers have initiated antitrust action against Google for AI Overviews not sending traffic back. Major publishers from AP to Sky News are all piling in to support Cloudflare’s anti-scraping technology, which will ban AI platforms from training their models on publisher sites or grabbing the latest stories as they are published.
Whilst this show of common thought is pleasing, a quote by one of the US Founding Fathers, (and newspaper publisher) Benjamin Franklin, comes to mind: “We must, indeed, all hang together, or must assuredly, we shall all hang separately”.
Franklin, speaking at the signing of the Declaration of Independence in 1776, was referring to his view that the US colonies must unite to have any chance of independence and act as one nation, or else face inevitable defeat by the British in the Revolutionary War.
My view is that the Franklin quote is getting more and more relevant in this era of news publishing. The web has already atomised what used to be small walled gardens where the readers and viewers were served a well monetised mix of reporting, general information and, yes, ads.
The result? The modern-day web has devastated journalism and the very concepts of truth and common understanding of facts in our societies. Things are bad. And now, it is about to become worse. Much worse.
Publishing faces an existential crisis
Until now, publishers could produce clickbait-style content that drew traffic that they could then monetise in search and social media. That era of cheap media outcomes is over.
The impact of LLMs means we have entered the zero-click era, where publishers lose a majority of their web traffic. Something like 60-70% of all open web traffic comes from blue link search, mainly Google. There won’t be much left of many newsrooms when they lose that traffic.
Cloudflare is leading the charge in trying to solve the problem of AI/LLM platforms not sending traffic to publishers. They plan to lock the content down so AI’s can’t just scrape it and incorporate it into the answers. The AI platforms would have to agree to be charged for any content they use. Major publishers such as Associated Press, The Atlantic, Buzzfeed, Conde Nast, DMGT, Dotdash Meredith, Fortune, Gannett, The Independent, Sky News, Time and Ziff Davis are backing Cloudflare.
But blocking will not fix the problem of traffic going away, and few publishers have the gravitas to strike a meaningful licensing deal. And remember, most of all, the content published has already been scraped.
Only a small cohort of publishers will have unique reporting that interests the AI/LLM users enough for them to want to pay for it. And if they do, they get paid for that crawl then. This will amount to peanuts in terms of revenue earned from original reporting.
Perhaps the answer lies in listening to Franklin
Over the years, I have been involved in various attempts by publishers, text and TV, and in more than one country, to collaborate to have a fighting chance against the web aggregators and walled gardens. Pool inventory, pool data, pool measurement – and become a must-buy and a must-have.
None of it has ever come off, and endless petty squabbles, rivalries and complications behind the scenes have marked these efforts. Now, there exists a point in time where publishers have to be relevant in the AI/LLM world. They can be scraped, crawled and reduced to serfs separately, or they can band together.
I believe they should form coalitions in their countries, markets or verticals. If all news publishers of a country join together, they become a must-have license for AI/LLM platforms that want to serve their users.
They should form publisher coalitions for their valuable content, and also promote their targeting and measurement abilities. Depending on the jurisdiction, such an initiative would be spearheaded by industry bodies, third-party consortia or some other model. Ironically, competition law in this space is geared to kill diversity by making this sort of collaboration hard. But the relevant competitive market is no longer the big local publishers against each other. It is the locals against the Big Tech platforms.
They must fight their own Revolutionary War and create a destination that users will go to because all the information they want is there, and they are a source of data that all others will need.
Behind the walls of their garden, they can use the ensuing lift in media yield to invest in the only thing that will be valuable in the AI era—more fresh proprietary information. A virtuous circle is possible here.
Martin Bertilsson is a Singapore-based tech entrepreneur and former Google ad tech director, now leading Multipole AI and building ventures at the intersection of AI and media.
(Except for the headline, this story has not been edited by PostX News and is published from a syndicated feed.)
