Crawlers—even the honest ones—have been spoofing browser User-Agent strings (partially or completely) since the beginning of the commercial Internet. Why would they do this? To dodge bad browser-sniffing scripts, which I still run into from time to time.
sure, but is that implying that 5% of this traffic is bad faith or rogue agents? the tools built into Claude Code and Codex surely have their own user-agent settings.
and why would they spoof Linux instead of Windows?
sure, but is that implying that 5% of this traffic is bad faith or rogue agents?
Given the volume of requests that have been semi-DDOS’ing a lot of servers? 5% is probably an underestimate.
the tools built into Claude Code and Codex surely have their own user-agent settings.
“Surely” is like “assume”—it says more about the speaker than the entity being discussed. They almost certainly don’t use completely distinct strings (no one does—they all start with “Mozilla/5.0”); at most they might have tacked an additional bit on at the tail. And that’s if they’re being aboveboard.
and why would they spoof Linux instead of Windows?
Why not? But actually, this may be due to an incorrect assumption on the part of the stats collectors. Typical strings for bots that admit to being bots don’t specify an OS: “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot” as opposed to, say “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 Edg/134.0.0.0” or “Mozilla/5.0 (Android 15; Mobile; SM-G556B/DS; rv:130.0) Gecko/130.0 Firefox/130.0”. If they’re dumping anything without OS markers into the “Linux” bucket, that might do it.
Crawlers—even the honest ones—have been spoofing browser
User-Agentstrings (partially or completely) since the beginning of the commercial Internet. Why would they do this? To dodge bad browser-sniffing scripts, which I still run into from time to time.sure, but is that implying that 5% of this traffic is bad faith or rogue agents? the tools built into Claude Code and Codex surely have their own
user-agentsettings.and why would they spoof Linux instead of Windows?
Given the volume of requests that have been semi-DDOS’ing a lot of servers? 5% is probably an underestimate.
“Surely” is like “assume”—it says more about the speaker than the entity being discussed. They almost certainly don’t use completely distinct strings (no one does—they all start with “Mozilla/5.0”); at most they might have tacked an additional bit on at the tail. And that’s if they’re being aboveboard.
Why not? But actually, this may be due to an incorrect assumption on the part of the stats collectors. Typical strings for bots that admit to being bots don’t specify an OS: “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot” as opposed to, say “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 Edg/134.0.0.0” or “Mozilla/5.0 (Android 15; Mobile; SM-G556B/DS; rv:130.0) Gecko/130.0 Firefox/130.0”. If they’re dumping anything without OS markers into the “Linux” bucket, that might do it.