Robots.txt

Robots.txt Rules

Crawler Rules
All
  • Allow: /
  • Disallow: /api/
  • Disallow: /*/files/
  • Disallow: /media

Sitemaps

Priority Location
1https://developer.mozilla.org/sitemap.xml

Raw Text

User-agent: *
Sitemap: https://developer.mozilla.org/sitemap.xml

Disallow: /api/
Disallow: /*/files/
Disallow: /media