Robots.txt

Robots.txt Rules

Crawler Rules
All
  • Allow: /
  • Allow: /wp-admin/admin-ajax.php
  • Allow: /wp-admin/load-scripts.php
  • Allow: /wp-admin/load-styles.php
  • Allow: /
  • Allow: /
  • Disallow: /wp-admin/
  • Disallow: /search
  • Disallow: /?s=
  • Disallow: /plugins/search/
gptbot
  • Allow: /
  • Allow: /
claudebot
  • Allow: /
  • Allow: /
anthropic-ai
  • Allow: /
  • Allow: /
google-extended
  • Allow: /
  • Allow: /
applebot-extended
  • Allow: /
  • Allow: /
perplexitybot
  • Allow: /
  • Allow: /
bytespider
  • Allow: /
  • Allow: /
ccbot
  • Allow: /
  • Allow: /
copilot
  • Allow: /
  • Allow: /

Sitemaps

Priority Location
9https://wordpress.org/sitemap.xml
8https://wordpress.org/news-sitemap.xml
7https://wordpress.org/themes/sitemap.xml
6https://wordpress.org/plugins/sitemap.xml
5https://wordpress.org/news/sitemap.xml
4https://wordpress.org/showcase/sitemap.xml
3https://wordpress.org/documentation/sitemap.xml
2https://wordpress.org/patterns/sitemap.xml
1https://wordpress.org/photos/sitemap.xml

Raw Text

Sitemap: https://wordpress.org/sitemap.xml
Sitemap: https://wordpress.org/news-sitemap.xml
Sitemap: https://wordpress.org/themes/sitemap.xml
Sitemap: https://wordpress.org/plugins/sitemap.xml
Sitemap: https://wordpress.org/news/sitemap.xml
Sitemap: https://wordpress.org/showcase/sitemap.xml
Sitemap: https://wordpress.org/documentation/sitemap.xml
Sitemap: https://wordpress.org/patterns/sitemap.xml
Sitemap: https://wordpress.org/photos/sitemap.xml
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Allow: /wp-admin/load-scripts.php
Allow: /wp-admin/load-styles.php

User-agent: *
Disallow: /search
Disallow: /?s=

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Bytespider
Allow: /

User-agent: CCBot
Allow: /

User-agent: Copilot
Allow: /

User-agent: *
Disallow: /plugins/search/