Robots.txt

Robots.txt Rules

Crawler Rules
All
  • Allow: /
  • Allow: /dist/latest/
  • Allow: /dist/latest/docs/api/
  • Allow: /api/
  • Disallow: /dist/
  • Disallow: /docs/

Sitemaps

Priority Location
2https://nodejs.org/sitemap.xml
1https://nodejs.org/learn/sitemap.xml

Raw Text

User-Agent: *
Allow: /dist/latest/
Allow: /dist/latest/docs/api/
Allow: /api/
Disallow: /dist/
Disallow: /docs/

Sitemap: https://nodejs.org/sitemap.xml
Sitemap: https://nodejs.org/learn/sitemap.xml