Undercover AI Agents · watching one file

nytimes.com
robots.txt, over time.

Currently blocks 16 of the agents on the register: OAI-SearchBot, ChatGPT-User, GPTBot, Claude-SearchBot, Claude-User, ClaudeBot, Perplexity-User, PerplexityBot, Meta-ExternalAgent, Bytespider, CCBot, Applebot, cohere-ai, Diffbot, Timpibot, omgilibot. Last read Sat, 12 Sep 2026 10:23 UTC; watched since Fri, 04 Sep 2026 16:58 UTC.

Follow this page as a feed: /watch/nytimes.com/feed.xml — works in any feed reader, Slack’s /feed, or an automation. Or get it by email.

Changes

1. No change recorded yet. The first read was Fri, 04 Sep 2026 16:58 UTC; each read since has found the same policy.

Read the file as it stands · Check another domain