By Web Publishers Unite Behind New Licensing Standard to Regulate AI Training Access
A New Wave of Licensing: Major Publishers Join Forces with RSL
Big names in digital publishing, including Reddit, Yahoo, Medium, Quora, and People Inc., have rallied behind a new framework – Really Simple Licensing (RSL). This open standard aims to grant content creators and site owners a certain degree of control over how their work is utilized by AI systems for training sessions. But this is just the tip of the iceberg.
Redefining Web Perception: RSL’s Impact on AI Access and Compensation
In a world that is increasingly digitized, RSL aspires to build upon the existing robots.txt protocol, an age-old tool that delineates what elements of a website are off-limits for web crawlers. But RSL is not just satisfied with the status quo. The framework provides provisions for publishers to define licensing and royalty terms, either through their robots.txt files or directly embedded in digital content, be it books, videos, or datasets.
This groundbreaking initiative comes from the RSL Collective, led by tech veterans Eckart Walther and Doug Leeds. Walther, the brain behind RSS, views RSL as a means to “create a new, scalable business model for the web.” On the other hand, Leeds, an ex-member of Ask.com and IAC Publishing, sees it akin to tried and tested models in the music industry.
What Lies Ahead: Challenges and Opportunities
The RSL platform brings in a host of flexible licensing models. Publishers are free to provide content gratis, or opt for payments via subscriptions, pay-per-crawl fees, or an innovative pay-per-inference model where the compensation links to an AI model citing their work.
However, no great reward comes without significant challenge. RSL’s fortune relies heavily on AI companies willingly joining the system. Historically, AI developers have been notorious for neglecting robots.txt directives, and monitoring usage for inference-based compensation gets intricate without cooperation. Leeds insinuates, “Our job is to get a big group of people to say it’s in your interest… because if you don’t, you’re violating everybody at once.”
Addressing content access by non-compliant bots, RSL is partnering with content delivery network Fastly to restrict bot access to protected content unless licensing terms are accepted. For non-Fastly users, enforcing control remains a challenge till a more comprehensive infrastructure is developed.
Despite legal uncertainties and potential lawsuits, Leeds stands strong, proposing that the RSL Collective leverages the power of law to protect licenses, in a manner similar to music rights organizations like ASCAP. Pooled legal costs could offer protection against unauthorized AI scraping.
A significant change ushered in by RSL is transparency. Leeds and Walther state, “There has always been a question of whether bots have agreed to terms that they don’t see… RSL changes that fundamentally, putting crawlers on notice of what the terms are before they access a site.”
Leeds drives home the point that RSL isn’t about creating something revolutionary but rather taking proven systems and deploying them in the digital world. “We’re bringing proven systems to a place where they haven’t existed before…because they haven’t had a standard that we could build on.” he says.
Joining the RSL Collective is free, and it already hosts familiar brands like O’Reilly, wikiHow, and Ziff Davis, the owner of IGN. Whether RSL becomes an industry standard or not remains to be seen and will depend largely on AI companies’ willingness to participate. However, at present, it paints a promising picture of equitable content usage on the web.
