source: techcrunch ai: patreon stops asking ai bots not to scrape — and starts blocking them
level: business
patreon is now directly blocking ai bots that scrape creator content for model training, moving beyond polite requests in robots.txt files. the membership platform partnered with cloudflare to use its ai crawl control technology, which actively prevents ai training crawlers from accessing the site. this change follows tests showing that individual crawlers went from thousands of weekly attempts to zero after blocking was enabled, indicating that many bots ignored the previous robots.txt instructions.
the company had already locked most paid content behind a paywall, but newer discovery features like a redesigned home feed and short-form quips risked exposing more material to crawlers. patreon will still allow bots that index pages for search and discovery, which can drive traffic back to the platform. the move reflects a broader push by online publishers to control how ai companies use their data, with cloudflare recently introducing tools like a pay-per-crawl marketplace and default blocking of mixed-use crawlers on ad-supported pages.
patreon's product chief emphasized that creators should have a say in how their work is used by ai, rather than being forced to accept training as a condition of reaching an audience. the company's approach contrasts with much of the internet, where opting out of ai scraping is often impractical. by enforcing technical blocks instead of relying on voluntary compliance, patreon aims to give creators more control over their intellectual property.
why it matters: this shows a practical shift from passive opt-out signals to active enforcement against unauthorized ai training data collection, relevant for data scientists and ai practitioners who rely on web-scraped datasets.
source: techcrunch ai: patreon stops asking ai bots not to scrape — and starts blocking them