meta-externalagent
An automatic crawler that can be allowed or disallowed through robots.txt.
Meta crawler controls
Meta indexing and user fetches use separate agents.
Meta says meta-externalagent crawls for use cases such as AI-model training or product improvement. meta-externalfetcher handles user-initiated link fetches and may bypass robots.txt.
meta-externalfetcher
A user-initiated fetcher. Meta explicitly warns that it may bypass robots.txt.
Request verification
Meta documents its crawler user agents and publishes network information for verifying traffic.
Block indexing example
Source: Meta web crawler documentation.