From Dev.to
How to charge AI crawlers per page with HTTP 402 (and keep your site free for people)
If you run a website, a growing share of your traffic isn't people. It's AI crawlers: GPTBot, ClaudeBot, PerplexityBot, Bytespider and a long tail of others, fetching pages to train models or answer q...