Whenever I see similar scanning in my server logs, I feel the urge to... provide them with same fake/poisoned credentials to... well, that's the problem. There is no way to hurt those. Usually I report the scan to Google (as they most often come from GCP IPs), but never hear back and it's not like it would stop them (it's some stolen cards or Google accounts, anyway, right?). It's just... it offends me there is no way to fight back.
Interesting that in some cases they are using the AI Agent crawler names to hide themselves. With the explosion in bot traffic it gets much easier to hide yourself in the crowd.
Whenever I see similar scanning in my server logs, I feel the urge to... provide them with same fake/poisoned credentials to... well, that's the problem. There is no way to hurt those. Usually I report the scan to Google (as they most often come from GCP IPs), but never hear back and it's not like it would stop them (it's some stolen cards or Google accounts, anyway, right?). It's just... it offends me there is no way to fight back.
Some useful information, though I'd like more details on headers sent like Sec-Fetch.
For passive blocking, TLSv1.3 filters out some bad bots and scrapers.
Most bad bots follow 302 redirect, but don't parse html and act on meta refresh.
Most bad bots don't store cookies, but I've seen a couple that do.
Interesting that in some cases they are using the AI Agent crawler names to hide themselves. With the explosion in bot traffic it gets much easier to hide yourself in the crowd.
I wonder if Cloudflare blocks it through their default protection?
I'm ok with having agents / LLMs reading my site, but not happy about scanners.