3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 2 days agoCloudflare blocking AI crawlerslemmy.relayeasy.comexternal-linkmessage-square33fedilinkarrow-up1345arrow-down13file-text
arrow-up1342arrow-down1external-linkCloudflare blocking AI crawlerslemmy.relayeasy.com3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 2 days agomessage-square33fedilinkfile-text
Cloudflare trying to stop AI crawling somehow! https://arstechnica.com/tech-policy/2025/07/pay-up-or-stop-scraping-cloudflare-program-charges-bots-for-each-crawl/
minus-squareProntomomo@lemmy.worldlinkfedilinkEnglisharrow-up5·2 days agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling
I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling