It reads the public robots.txt policy, documented crawler tokens, discovery files, technical responses and indexability signals. A verdict is evidence about the tested URL at a point in time, not a promise of visits or citations.
Can I allow AI search but block training?
Often, yes. Search, training and user-triggered retrieval can use different user-agent tokens. Review each provider’s current documentation and publish explicit groups rather than one blanket rule.
Does robots.txt protect private content?
No. robots.txt communicates preferences to compliant crawlers. Use authentication and authorization for private or sensitive material.
Is the audit free and does it need an API key?
The public checker is designed for quick, keyless checks. It fetches public evidence and does not replace server logs, Search Console or a legal review.
