OAI-SearchBot
Search crawler. Blocking it opts the site out of ChatGPT search answers, though navigational links may still appear.
OpenAI crawler controls
OAI-SearchBot is used to surface pages in ChatGPT search. GPTBot crawls content that may be used to improve OpenAI foundation models. OpenAI documents these as independent settings.
Search crawler. Blocking it opts the site out of ChatGPT search answers, though navigational links may still appear.
Model-development crawler. Disallowing it indicates that crawled content should not be used for training generative AI foundation models.
User-initiated fetcher. It is not an automatic web crawler, and OpenAI says robots.txt rules may not apply.
Allow search discovery while blocking model-development crawling.
Source: OpenAI crawler documentation. User-agent versions may change; use the product tokens in robots.txt.