Robots.txt

Robots.txt Rules

Crawler Rules
All
  • Allow: /
  • Disallow: /users/*?
  • Disallow: /join/*?
  • Disallow: /download/
  • Disallow: /checkout/
  • Disallow: /global/
  • Disallow: /api/
  • Disallow: /_nsfgfb/
  • Disallow: /*?*cursor=
pinterestbot
  • Allow: /
  • Disallow: /

Sitemaps

Priority Location
1https://www.deviantart.com/sitemap-index.xml.gz

Raw Text

User-agent: *
Disallow: /users/*?
Disallow: /join/*?
Disallow: /download/
Disallow: /checkout/
Disallow: /global/
Disallow: /api/
Disallow: /_nsfgfb/
Disallow: /*?*cursor=
Crawl-delay: 1

User-agent: Pinterestbot
Disallow: /
 
Sitemap: https://www.deviantart.com/sitemap-index.xml.gz