robots.txt blocks Meta-ExternalAgent (Meta AI / Llama training)
Meta-ExternalAgent is the crawler Meta uses to gather content for training its Llama AI models and powering Meta AI, the assistant built into WhatsApp, Instagram and Facebook. This check found robots.txt disallowing Meta-ExternalAgent. That block applies across the board, so none of Meta's AI products can currently draw on your pages.
Why this matters
Meta-ExternalAgent is a crawler Meta documents for AI-related use. Blocking it may stop Meta from using your content for those purposes. This is a policy choice; it may matter more for consumer-facing businesses whose audience is active on Meta's platforms, and less for others.
How to fix it
Your developer needs about 5 minutes. Add an allow rule for Meta-ExternalAgent in robots.txt, unless you're deliberately opting out of Llama training, in which case document that decision in your llms.txt file. Meta AI's reach through WhatsApp and Instagram makes this a higher-value fix than it might first appear for consumer-facing brands.
Turn website findings into an action plan
Create an account to start a Full Audit of your website, with supporting evidence and a prioritised action plan.
Start Full Audit