Robots.txt

Robots.txt Rules

Crawler Rules
All
  • Allow: /
  • Disallow: /wp-admin/
  • Disallow: /wp-includes/
  • Disallow: /*?*ajax=
  • Disallow: /search/

Sitemaps

Priority Location
2https://www.engadget.com/sitemap_index.xml
1https://www.engadget.com/?getfeed=google

Raw Text

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /*?*ajax=
Disallow: /search/

Sitemap: https://www.engadget.com/sitemap_index.xml
Sitemap: https://www.engadget.com/?getfeed=google