Governance files explained

A plain-language overview of each governance file type and why it matters. See the Governance Checks section for detailed technical reference.

The essentials

• robots.txt — tells search engines what to crawl (critical for SEO)
• XML sitemaps — tells search engines what pages exist (critical for SEO)
• security.txt — tells researchers how to report vulnerabilities (critical for security)

Important for advertising

• ads.txt — prevents ad fraud by declaring authorised sellers

Emerging standards

• llms.txt — controls AI crawler interaction with your content

Nice to have

• humans.txt — credits the team behind the site
• Cookies & consent — compliance with privacy regulations