Robots.txt

Robots.txt Rules

Crawler	Rules
All	Allow: /
oai-searchbot	Allow: /
chatgpt-user	Allow: /
chatgpt agent	Allow: /
claude-user	Allow: /
claude-searchbot	Allow: /
gemini-deep-research	Allow: /
google-notebooklm	Allow: /
applebot	Allow: /
meta-externalfetcher	Allow: / Allow: /
meta-webindexer	Allow: /
facebookexternalhit	Allow: /
amazonbot	Allow: /
perplexitybot	Allow: /
perplexity-user	Allow: /
mistralai-user	Allow: /
mistralai-user/1.0	Allow: /
duckassistbot	Allow: /
bravebot	Allow: /
youbot	Allow: / Allow: /?autoplay Disallow: /account/ Disallow: /support/ Disallow: /widget/ Disallow: /xml/ Disallow: /landing/ Disallow: //landing/ Disallow: /search/ Disallow: //search/ Disallow: checkForm= Disallow: /plugins/ Disallow: /profile/ Disallow: //profile/ Disallow: /video/ Disallow: //video/ Disallow: //acq Disallow: //naboo Disallow: /artist/127231622/radio?autoplay=true Disallow: /*?
twitterbot	Allow: / Disallow:
gptbot	Allow: /
anthropic-ai	Allow: /
claudebot	Allow: /
claude-web	Allow: /
google-extended	Allow: /
googleother	Allow: /
applebot-extended	Allow: /
meta-externalagent	Allow: / Allow: /
facebookbot	Allow: /
cohere-ai	Allow: /
cohere-training-data-crawler	Allow: /
bytespider	Allow: /
tiktokspider	Allow: /
deepseekbot	Allow: /
ai2bot	Allow: /
ai2bot-dolma	Allow: /
ccbot	Allow: /
yandexadditionalbot	Allow: /
diffbot	Allow: /
omgili	Allow: /
omgilibot	Allow: /
webzio-extended	Allow: / Allow: /
laiondownloader	Allow: /
img2dataset	Allow: /
firecrawlagent	Allow: / Disallow: //artist/ Disallow: //album/ Disallow: //track/ Disallow: //playlist/ Disallow: /*/channels/podcasts

Sitemaps

Priority	Location
2	https://www.deezer.com/sitemap.xml
1	https://www.deezer.com/catalog_sitemap.xml

Raw Text

User-agent: *

# --- OpenAI ---
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: ChatGPT Agent

# --- Anthropic ---
User-agent: Claude-User
User-agent: Claude-SearchBot

# --- Google ---
User-agent: Gemini-Deep-Research
User-agent: Google-NotebookLM

# --- Apple ---
User-agent: Applebot

# --- Meta ---
User-agent: Meta-ExternalFetcher
User-agent: meta-externalfetcher
User-agent: meta-webindexer
User-agent: facebookexternalhit

# --- Amazon ---
User-agent: Amazonbot

# --- Perplexity ---
User-agent: PerplexityBot
User-agent: Perplexity-User

# --- Mistral ---
User-agent: MistralAI-User
User-agent: MistralAI-User/1.0

# --- DuckDuckGo ---
User-agent: DuckAssistBot

# --- Brave ---
User-agent: Bravebot

# --- You.com ---
User-agent: YouBot

Disallow: /account/
Disallow: /support/
Disallow: /widget/
Disallow: /xml/

Disallow: /landing/*
Disallow: /*/landing/*
Disallow: /search/*
Disallow: /*/search/*

Disallow: *checkForm=*
Disallow: /plugins/*

Disallow: /profile/*
Disallow: /*/profile/*
Disallow: /video/*
Disallow: /*/video/*

Disallow: /*/acq

Disallow: /*/naboo

Disallow: /artist/127231622/radio?autoplay=true
Allow: /*?autoplay
Disallow: /*?

User-agent: Twitterbot
Disallow:

User-agent: GPTBot
User-agent: anthropic-ai
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: Google-Extended
User-agent: GoogleOther
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
User-agent: meta-externalagent
User-agent: FacebookBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: Bytespider
User-agent: TikTokSpider
User-agent: DeepSeekBot
User-agent: AI2Bot
User-agent: Ai2Bot-Dolma
User-agent: CCBot
User-agent: YandexAdditionalBot
User-agent: Diffbot
User-agent: omgili
User-agent: omgilibot
User-agent: Webzio-Extended
User-agent: webzio-extended
User-agent: LAIONDownloader
User-agent: img2dataset
User-agent: FirecrawlAgent
Disallow: /*/artist/
Disallow: /*/album/
Disallow: /*/track/
Disallow: /*/playlist/
Disallow: /*/channels/podcasts

Sitemap: https://www.deezer.com/sitemap.xml
Sitemap: https://www.deezer.com/catalog_sitemap.xml