

yes, you can match on user agent, and then conditionally serve them other stuff (most webservers are fine with this). nepenthes and iocaine are the current preferred/recommended servers to serve them bot mazes
the thing is that the crawlers will also lie (openai definitely doesn’t publish all its own source IPs, I’ve verified this myself), and will attempt a number of workarounds (like using residential proxies too)
the term of art is “residential proxy” and there’s a ton of them
for example: it’s the flipside of Bright’s free VPN service - through Bright Data they sell people access proxied via some user’s connection