Robots.txt

Robots.txt Rules

Crawler Rules
facebookexternalhit
  • Allow: /
  • Allow: /
bytespider
  • Allow: /
  • Disallow: /
Baidu
  • Allow: /
  • Disallow: /
petalbot
  • Allow: /
  • Disallow: /
tiktokspider
  • Allow: /
  • Disallow: /
twitterbot
  • Allow: /
  • Allow: /*?utm_
Bing
  • Allow: /
  • Disallow: /?s=
  • Disallow: /search/
All
  • Allow: /
  • Allow: /
  • Allow: /*?cexp_id
  • Allow: /*?show=trending
  • Allow: /*?show=recent
  • Allow: /*wp-includes*.jpg
  • Allow: /*wp-content*.jpg
  • Allow: /*wp-includes*.jpeg
  • Allow: /*wp-content*.jpeg
  • Allow: /*wp-includes*.jpe
  • Allow: /*wp-content*.jpe
  • Allow: /*wp-includes*.png
  • Allow: /*wp-content*.png
  • Allow: /*wp-includes*.css
  • Allow: /*wp-content*.css
  • Allow: /*wp-includes*.js
  • Allow: /*wp-content*.js
  • Allow: /*wp-includes*.gif
  • Allow: /*wp-content*.gif
  • Allow: /*wp-includes*.svg
  • Allow: /*wp-content*.svg
  • Allow: /*wp-includes*.webp
  • Allow: /*wp-content*.webp
  • Disallow: /*wp-admin/
  • Disallow: /*wp-includes
  • Disallow: /*wp-content
  • Disallow: /*wp-json
  • Disallow: /contributor/

Sitemaps

Priority Location
6https://www.boredpanda.com/sitemap.xml
5https://www.boredpanda.com/tags-sitemap.xml
4https://www.boredpanda.com/categories-sitemap.xml
3https://www.boredpanda.com/news-sitemap.xml
2https://www.boredpanda.com/authors-sitemap.xml
1https://www.boredpanda.com/games-sitemap.xml

Raw Text

User-agent: facebookexternalhit
Allow: /

User-agent: Bytespider
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: TikTokSpider
Disallow: /

User-agent: Twitterbot
Allow: /*?utm_

User-agent: Bingbot
Disallow: /?s=
Disallow: /search/

User-agent: *
Allow: /
Disallow: /*wp-admin/
Disallow: /*wp-includes
Disallow: /*wp-content
Disallow: /*wp-json
Disallow: /contributor/

Allow: /*?cexp_id
Allow: /*?show=trending
Allow: /*?show=recent
Allow: /*wp-includes*.jpg
Allow: /*wp-content*.jpg
Allow: /*wp-includes*.jpeg
Allow: /*wp-content*.jpeg
Allow: /*wp-includes*.jpe
Allow: /*wp-content*.jpe
Allow: /*wp-includes*.png
Allow: /*wp-content*.png
Allow: /*wp-includes*.css
Allow: /*wp-content*.css
Allow: /*wp-includes*.js
Allow: /*wp-content*.js
Allow: /*wp-includes*.gif
Allow: /*wp-content*.gif
Allow: /*wp-includes*.svg
Allow: /*wp-content*.svg
Allow: /*wp-includes*.webp
Allow: /*wp-content*.webp


Sitemap: https://www.boredpanda.com/sitemap.xml
Sitemap: https://www.boredpanda.com/tags-sitemap.xml
Sitemap: https://www.boredpanda.com/categories-sitemap.xml
Sitemap: https://www.boredpanda.com/news-sitemap.xml
Sitemap: https://www.boredpanda.com/authors-sitemap.xml
Sitemap: https://www.boredpanda.com/games-sitemap.xml