Robots.txt

Robots.txt Rules

Crawler Rules
yandex
  • Allow: /
  • Allow: */uploads
  • Allow: /*/*.js
  • Allow: /*/*.css
  • Allow: /wp-*.png
  • Allow: /wp-*.jpg
  • Allow: /wp-*.jpeg
  • Allow: /wp-*.gif
  • Allow: /wp-*.svg
  • Allow: /wp-*.pdf
  • Allow: /wp-admin/admin-ajax.php
  • Allow: /feed/turbo
  • Disallow: /wp-admin/
  • Disallow: /wp-includes/
  • Disallow: /wp-content/plugins/
  • Disallow: /wp-content/cache/
  • Disallow: /wp-comments
  • Disallow: /wp-login.php
  • Disallow: /?s=
  • Disallow: /?p=*&preview=true
  • Disallow: /api/
  • Disallow: /wp-json/
  • Disallow: */amp/
  • Disallow: /wp-content/uploads/wp-import-export-lite/
Google
  • Allow: /
  • Allow: */uploads
  • Allow: /*/*.js
  • Allow: /*/*.css
  • Allow: /wp-*.png
  • Allow: /wp-*.jpg
  • Allow: /wp-*.jpeg
  • Allow: /wp-*.gif
  • Allow: /wp-*.svg
  • Allow: /wp-*.pdf
  • Allow: /wp-admin/admin-ajax.php
  • Disallow: /wp-admin/
  • Disallow: /wp-includes/
  • Disallow: /wp-content/plugins/
  • Disallow: /wp-content/cache/
  • Disallow: /wp-comments
  • Disallow: /wp-login.php
  • Disallow: /?s=
  • Disallow: /?p=*&preview=true
  • Disallow: /api/
  • Disallow: /wp-json/
  • Disallow: /wp-content/uploads/wp-import-export-lite/
All
  • Allow: /
  • Allow: */uploads
  • Allow: /*/*.js
  • Allow: /*/*.css
  • Allow: /wp-*.png
  • Allow: /wp-*.jpg
  • Allow: /wp-*.jpeg
  • Allow: /wp-*.gif
  • Allow: /wp-*.svg
  • Allow: /wp-*.pdf
  • Allow: /wp-admin/admin-ajax.php
  • Disallow: /wp-admin/
  • Disallow: /wp-includes/
  • Disallow: /wp-content/plugins/
  • Disallow: /wp-content/cache/
  • Disallow: /wp-comments
  • Disallow: /wp-login.php
  • Disallow: /?s=
  • Disallow: /?p=*&preview=true
  • Disallow: /api/
  • Disallow: /wp-json/
  • Disallow: /wp-content/uploads/wp-import-export-lite/
mediapartners-google
  • Allow: /
  • Disallow:

Sitemaps

Priority Location
1https://thecode.media/sitemap_index.xml

Raw Text

User-agent: Yandex
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-comments
Disallow: /wp-login.php
Disallow: /?s=
Disallow: /?p=*&preview=true
Disallow: /api/*
Disallow: /wp-json/
Disallow: */amp/
Disallow: /wp-content/uploads/wp-import-export-lite/
Allow: */uploads
Allow: /*/*.js
Allow: /*/*.css
Allow: /wp-*.png
Allow: /wp-*.jpg
Allow: /wp-*.jpeg
Allow: /wp-*.gif
Allow: /wp-*.svg
Allow: /wp-*.pdf
Allow: /wp-admin/admin-ajax.php
Allow: /feed/turbo*
Clean-param: fbclid&ref&utm_source&utm_campaign&utm_content&utm_medium&sa&_gl&epik&utm_compaign&_utm_source&_utm_campaign&ved&yhic&yqpil&yprqee&creative&noamp
Clean-param: from&back&_gclid&preview_id&&attachment_id&submit&utm_referrer&yrwinfo&id&_utm_medium&_utm_content&utm_reader&user_id&etext&ctime&clckid

User-agent: Googlebot
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-comments
Disallow: /wp-login.php
Disallow: /?s=
Disallow: /?p=*&preview=true
Disallow: /api/*
Disallow: /wp-json/
Disallow: /wp-content/uploads/wp-import-export-lite/
Allow: */uploads
Allow: /*/*.js
Allow: /*/*.css
Allow: /wp-*.png
Allow: /wp-*.jpg
Allow: /wp-*.jpeg
Allow: /wp-*.gif
Allow: /wp-*.svg
Allow: /wp-*.pdf
Allow: /wp-admin/admin-ajax.php

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-comments
Disallow: /wp-login.php
Disallow: /?s=
Disallow: /?p=*&preview=true
Disallow: /api/*
Disallow: /wp-json/
Disallow: /wp-content/uploads/wp-import-export-lite/
Allow: */uploads
Allow: /*/*.js
Allow: /*/*.css
Allow: /wp-*.png
Allow: /wp-*.jpg
Allow: /wp-*.jpeg
Allow: /wp-*.gif
Allow: /wp-*.svg
Allow: /wp-*.pdf
Allow: /wp-admin/admin-ajax.php

User-agent: Mediapartners-Google
Disallow:

Sitemap: https://thecode.media/sitemap_index.xml