Resident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 3 days agoAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comexternal-linkmessage-square15fedilinkarrow-up1131arrow-down13file-textcross-posted to: [email protected][email protected]
arrow-up1128arrow-down1external-linkAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comResident Pulser@infosec.pubB to Pulse of Truth@infosec.pubEnglish · 3 days agomessage-square15fedilinkfile-textcross-posted to: [email protected][email protected]
minus-squareBlaster M@lemmy.worldlinkfedilinkEnglisharrow-up5·2 days agoRather funny that some of these are in code if(useragent == chatgpt) kind of sillyness. You need heiuristics to detect scrapers, because they’ll just report their user agent as the average user’s browser.
Rather funny that some of these are in code if(useragent == chatgpt) kind of sillyness. You need heiuristics to detect scrapers, because they’ll just report their user agent as the average user’s browser.