Robots.txt

Robots.txt Rules

Crawler	Rules
All	Allow: / Disallow: /account/ Disallow: /auth/ Disallow: /signup Disallow: /works//carousel Disallow: /discover/ Disallow: /mybubble/ Disallow: /objections/ Disallow: /settings/ Disallow: /.json$ Disallow: /api/ Disallow: /journals$ Disallow: /people//journal/ Disallow: /groups/ Disallow: /shop?query= Disallow: /shop&query= Disallow: /modals/ Disallow: /typeahead/ Disallow: /fep/api/user-events Disallow: /fep/api/shop-bff/artist-badge Disallow: /fep/api/shop-bff/other-products Disallow: /fep/api/shop-bff/available-styles Disallow: /fep/api/shop-bff/artist-shop-info Disallow: /fep/api/shop-bff/artist-portfolio-items Disallow: /g/test- Disallow: /de/g/test- Disallow: /fr/g/test- Disallow: /en/g/test- Disallow: /es/g/test- Disallow: /i//sexy-bf Disallow: /i//Sexy-bf Disallow: /i//Sexy-Bf Disallow: /i//Sexy-BF Disallow: /i//bf-sexy Disallow: /i//bf-Sexy Disallow: /i//BF-Sexy Disallow: /i//sexy+bf Disallow: /i//Sexy+bf Disallow: /i//Sexy+Bf Disallow: /i//Sexy+BF Disallow: /i//bf+sexy Disallow: /i//bf+Sexy Disallow: /i//BF+Sexy Disallow: /i//sexybf Disallow: /i//bfsexy Disallow: /i//xxxxx Disallow: /i//english+bf Disallow: /i//English+BF Disallow: /i//English+Bf Disallow: /i//bangla+bf Disallow: /i//Bangla+BF Disallow: /i//Bangla+Bf Disallow: /i//bangladeshi+bf Disallow: /i//Bangladeshi+Bf Disallow: /i//Indian+BF Disallow: /i//INDIAN+BF Disallow: /i//indian+bf Disallow: /i//SEXY-BF Disallow: /i//sexy%20bf Disallow: /i//bf+hot+sexy Disallow: /i//XXXXX Disallow: /i//Xxxxx Disallow: /i//Sexy-Boyfriend-BF Disallow: /i//Bangladeshi-Bf Disallow: /i//Bangla-Bf Disallow: /i//Bangla-Local-Bf Disallow: /i//English-Bf Disallow: /shop/sexy-bf Disallow: /shop/Sexy-bf Disallow: /shop/Sexy-Bf Disallow: /shop/Sexy-BF Disallow: /shop/bf-sexy Disallow: /shop/bf-Sexy Disallow: /shop/BF-Sexy Disallow: /shop/sexy+bf Disallow: /shop/Sexy+bf Disallow: /shop/Sexy+Bf Disallow: /shop/Sexy+BF Disallow: /shop/bf+sexy Disallow: /shop/bf+Sexy Disallow: /shop/BF+Sexy Disallow: /shop/sexybf Disallow: /shop/bfsexy Disallow: /shop/xxxxx Disallow: /shop/bangla+bf Disallow: /shop/Bangla+BF Disallow: /shop/Bangla+Bf Disallow: /shop/bangladeshi+bf Disallow: /shop/Bangladeshi+Bf Disallow: /shop/english+bf Disallow: /shop/English+BF Disallow: /shop/English+Bf Disallow: /shop/SEXY-BF Disallow: /shop/sexy%20bf Disallow: /shop/Indian+BF Disallow: /shop/INDIAN+BF Disallow: /shop/indian+bf Disallow: /shop/bf+hot+sexy Disallow: /shop/XXXXX Disallow: /shop/Xxxxx Disallow: /shop/Sexy-Boyfriend-BF Disallow: /shop/Bangladeshi-Bf Disallow: /shop/Bangla-Bf Disallow: /shop/Bangla-Local-Bf Disallow: /shop/English-Bf
pinterestbot	Allow: / Allow: / Disallow: /account/ Disallow: /auth/ Disallow: /signup Disallow: /works//carousel Disallow: /discover/ Disallow: /mybubble/ Disallow: /objections/ Disallow: /settings/ Disallow: /.json$ Disallow: /api/ Disallow: /journals$ Disallow: /people//journal/ Disallow: /groups/ Disallow: /shop?query= Disallow: /shop&query= Disallow: /modals/ Disallow: /typeahead/ Disallow: /fep/api/user-events Disallow: /fep/api/shop-bff/artist-badge Disallow: /fep/api/shop-bff/other-products Disallow: /fep/api/shop-bff/available-styles Disallow: /fep/api/shop-bff/artist-shop-info Disallow: /fep/api/shop-bff/artist-portfolio-items Disallow: /g/test- Disallow: /de/g/test- Disallow: /fr/g/test- Disallow: /en/g/test- Disallow: /es/g/test- Disallow: /i//sexy-bf Disallow: /i//Sexy-bf Disallow: /i//Sexy-Bf Disallow: /i//Sexy-BF Disallow: /i//bf-sexy Disallow: /i//bf-Sexy Disallow: /i//BF-Sexy Disallow: /i//sexy+bf Disallow: /i//Sexy+bf Disallow: /i//Sexy+Bf Disallow: /i//Sexy+BF Disallow: /i//bf+sexy Disallow: /i//bf+Sexy Disallow: /i//BF+Sexy Disallow: /i//sexybf Disallow: /i//bfsexy Disallow: /i//xxxxx Disallow: /i//english+bf Disallow: /i//English+BF Disallow: /i//English+Bf Disallow: /i//bangla+bf Disallow: /i//Bangla+BF Disallow: /i//Bangla+Bf Disallow: /i//bangladeshi+bf Disallow: /i//Bangladeshi+Bf Disallow: /i//Indian+BF Disallow: /i//INDIAN+BF Disallow: /i//indian+bf Disallow: /i//SEXY-BF Disallow: /i//sexy%20bf Disallow: /i//bf+hot+sexy Disallow: /i//XXXXX Disallow: /i//Xxxxx Disallow: /i//Sexy-Boyfriend-BF Disallow: /i//Bangladeshi-Bf Disallow: /i//Bangla-Bf Disallow: /i//Bangla-Local-Bf Disallow: /i//English-Bf Disallow: /shop/sexy-bf Disallow: /shop/Sexy-bf Disallow: /shop/Sexy-Bf Disallow: /shop/Sexy-BF Disallow: /shop/bf-sexy Disallow: /shop/bf-Sexy Disallow: /shop/BF-Sexy Disallow: /shop/sexy+bf Disallow: /shop/Sexy+bf Disallow: /shop/Sexy+Bf Disallow: /shop/Sexy+BF Disallow: /shop/bf+sexy Disallow: /shop/bf+Sexy Disallow: /shop/BF+Sexy Disallow: /shop/sexybf Disallow: /shop/bfsexy Disallow: /shop/xxxxx Disallow: /shop/bangla+bf Disallow: /shop/Bangla+BF Disallow: /shop/Bangla+Bf Disallow: /shop/bangladeshi+bf Disallow: /shop/Bangladeshi+Bf Disallow: /shop/english+bf Disallow: /shop/English+BF Disallow: /shop/English+Bf Disallow: /shop/SEXY-BF Disallow: /shop/sexy%20bf Disallow: /shop/Indian+BF Disallow: /shop/INDIAN+BF Disallow: /shop/indian+bf Disallow: /shop/bf+hot+sexy Disallow: /shop/XXXXX Disallow: /shop/Xxxxx Disallow: /shop/Sexy-Boyfriend-BF Disallow: /shop/Bangladeshi-Bf Disallow: /shop/Bangla-Bf Disallow: /shop/Bangla-Local-Bf Disallow: /shop/English-Bf
Bing	Allow: / Allow: /

Sitemaps

Priority	Location
Nothing to show.

Raw Text

User-agent:*
Disallow: /account/
Disallow: /auth/
Disallow: /signup
Disallow: */works/*/carousel
Disallow: /discover/
Disallow: /mybubble/
Disallow: /objections/
Disallow: /settings/
Disallow: /*.json$
Disallow: /api/
Disallow: /journals$
Disallow: /people/*/journal/
Disallow: /groups/*
Disallow: /shop*?query=*
Disallow: /shop*&query=*
Disallow: /modals/*
Disallow: /typeahead/
Disallow: /fep/api/user-events
Disallow: /fep/api/shop-bff/artist-badge
Disallow: /fep/api/shop-bff/other-products
Disallow: /fep/api/shop-bff/available-styles
Disallow: /fep/api/shop-bff/artist-shop-info
Disallow: /fep/api/shop-bff/artist-portfolio-items
Disallow: /g/test-*
Disallow: /de/g/test-*
Disallow: /fr/g/test-*
Disallow: /en/g/test-*
Disallow: /es/g/test-*

# Disallow low-value pages attracting bad traffic for DEP-3243
Disallow: /i/*/*sexy-bf*
Disallow: /i/*/*Sexy-bf*
Disallow: /i/*/*Sexy-Bf*
Disallow: /i/*/*Sexy-BF*
Disallow: /i/*/*bf-sexy*
Disallow: /i/*/*bf-Sexy*
Disallow: /i/*/*BF-Sexy*
Disallow: /i/*/*sexy+bf*
Disallow: /i/*/*Sexy+bf*
Disallow: /i/*/*Sexy+Bf*
Disallow: /i/*/*Sexy+BF*
Disallow: /i/*/*bf+sexy*
Disallow: /i/*/*bf+Sexy*
Disallow: /i/*/*BF+Sexy*
Disallow: /i/*/*sexybf*
Disallow: /i/*/*bfsexy*
Disallow: /i/*/*xxxxx*
Disallow: /i/*/*english+bf*
Disallow: /i/*/*English+BF*
Disallow: /i/*/*English+Bf*
Disallow: /i/*/*bangla+bf*
Disallow: /i/*/*Bangla+BF*
Disallow: /i/*/*Bangla+Bf*
Disallow: /i/*/*bangladeshi+bf*
Disallow: /i/*/*Bangladeshi+Bf*
Disallow: /i/*/*Indian+BF*
Disallow: /i/*/*INDIAN+BF*
Disallow: /i/*/*indian+bf*
Disallow: /i/*/*SEXY-BF*
Disallow: /i/*/*sexy%20bf*
Disallow: /i/*/*bf+hot+sexy*
Disallow: /i/*/*XXXXX*
Disallow: /i/*/*Xxxxx*
Disallow: /i/*/*Sexy-Boyfriend-BF*
Disallow: /i/*/*Bangladeshi-Bf*
Disallow: /i/*/*Bangla-Bf*
Disallow: /i/*/*Bangla-Local-Bf*
Disallow: /i/*/*English-Bf*
Disallow: /shop/*sexy-bf*
Disallow: /shop/*Sexy-bf*
Disallow: /shop/*Sexy-Bf*
Disallow: /shop/*Sexy-BF*
Disallow: /shop/*bf-sexy*
Disallow: /shop/*bf-Sexy*
Disallow: /shop/*BF-Sexy*
Disallow: /shop/*sexy+bf*
Disallow: /shop/*Sexy+bf*
Disallow: /shop/*Sexy+Bf*
Disallow: /shop/*Sexy+BF*
Disallow: /shop/*bf+sexy*
Disallow: /shop/*bf+Sexy*
Disallow: /shop/*BF+Sexy*
Disallow: /shop/*sexybf*
Disallow: /shop/*bfsexy*
Disallow: /shop/*xxxxx*
Disallow: /shop/*bangla+bf*
Disallow: /shop/*Bangla+BF*
Disallow: /shop/*Bangla+Bf*
Disallow: /shop/*bangladeshi+bf*
Disallow: /shop/*Bangladeshi+Bf*
Disallow: /shop/*english+bf*
Disallow: /shop/*English+BF*
Disallow: /shop/*English+Bf*
Disallow: /shop/*SEXY-BF*
Disallow: /shop/*sexy%20bf*
Disallow: /shop/*Indian+BF*
Disallow: /shop/*INDIAN+BF*
Disallow: /shop/*indian+bf*
Disallow: /shop/*bf+hot+sexy*
Disallow: /shop/*XXXXX*
Disallow: /shop/*Xxxxx*
Disallow: /shop/*Sexy-Boyfriend-BF*
Disallow: /shop/*Bangladeshi-Bf*
Disallow: /shop/*Bangla-Bf*
Disallow: /shop/*Bangla-Local-Bf*
Disallow: /shop/*English-Bf*

User-agent: Pinterestbot
Crawl-delay: 0.2

User-agent: Bingbot
User-agent: Pinterestbot

# Duplicate of the wildcard rule above
Disallow: /account/
Disallow: /auth/
Disallow: /signup
Disallow: */works/*/carousel
Disallow: /discover/
Disallow: /mybubble/
Disallow: /objections/
Disallow: /settings/
Disallow: /*.json$
Disallow: /api/
Disallow: /journals$
Disallow: /people/*/journal/
Disallow: /groups/*
Disallow: /shop*?query=*
Disallow: /shop*&query=*
Disallow: /modals/*
Disallow: /typeahead/
Disallow: /fep/api/user-events
Disallow: /fep/api/shop-bff/artist-badge
Disallow: /fep/api/shop-bff/other-products
Disallow: /fep/api/shop-bff/available-styles
Disallow: /fep/api/shop-bff/artist-shop-info
Disallow: /fep/api/shop-bff/artist-portfolio-items
Disallow: /g/test-*
Disallow: /de/g/test-*
Disallow: /fr/g/test-*
Disallow: /en/g/test-*
Disallow: /es/g/test-*

# Disallow low-value pages attracting bad traffic for DEP-3243 & DEP-3502
Disallow: /i/*/*sexy-bf*
Disallow: /i/*/*Sexy-bf*
Disallow: /i/*/*Sexy-Bf*
Disallow: /i/*/*Sexy-BF*
Disallow: /i/*/*bf-sexy*
Disallow: /i/*/*bf-Sexy*
Disallow: /i/*/*BF-Sexy*
Disallow: /i/*/*sexy+bf*
Disallow: /i/*/*Sexy+bf*
Disallow: /i/*/*Sexy+Bf*
Disallow: /i/*/*Sexy+BF*
Disallow: /i/*/*bf+sexy*
Disallow: /i/*/*bf+Sexy*
Disallow: /i/*/*BF+Sexy*
Disallow: /i/*/*sexybf*
Disallow: /i/*/*bfsexy*
Disallow: /i/*/*xxxxx*
Disallow: /i/*/*english+bf*
Disallow: /i/*/*English+BF*
Disallow: /i/*/*English+Bf*
Disallow: /i/*/*bangla+bf*
Disallow: /i/*/*Bangla+BF*
Disallow: /i/*/*Bangla+Bf*
Disallow: /i/*/*bangladeshi+bf*
Disallow: /i/*/*Bangladeshi+Bf*
Disallow: /i/*/*Indian+BF*
Disallow: /i/*/*INDIAN+BF*
Disallow: /i/*/*indian+bf*
Disallow: /i/*/*SEXY-BF*
Disallow: /i/*/*sexy%20bf*
Disallow: /i/*/*bf+hot+sexy*
Disallow: /i/*/*XXXXX*
Disallow: /i/*/*Xxxxx*
Disallow: /i/*/*Sexy-Boyfriend-BF*
Disallow: /i/*/*Bangladeshi-Bf*
Disallow: /i/*/*Bangla-Bf*
Disallow: /i/*/*Bangla-Local-Bf*
Disallow: /i/*/*English-Bf*
Disallow: /shop/*sexy-bf*
Disallow: /shop/*Sexy-bf*
Disallow: /shop/*Sexy-Bf*
Disallow: /shop/*Sexy-BF*
Disallow: /shop/*bf-sexy*
Disallow: /shop/*bf-Sexy*
Disallow: /shop/*BF-Sexy*
Disallow: /shop/*sexy+bf*
Disallow: /shop/*Sexy+bf*
Disallow: /shop/*Sexy+Bf*
Disallow: /shop/*Sexy+BF*
Disallow: /shop/*bf+sexy*
Disallow: /shop/*bf+Sexy*
Disallow: /shop/*BF+Sexy*
Disallow: /shop/*sexybf*
Disallow: /shop/*bfsexy*
Disallow: /shop/*xxxxx*
Disallow: /shop/*bangla+bf*
Disallow: /shop/*Bangla+BF*
Disallow: /shop/*Bangla+Bf*
Disallow: /shop/*bangladeshi+bf*
Disallow: /shop/*Bangladeshi+Bf*
Disallow: /shop/*english+bf*
Disallow: /shop/*English+BF*
Disallow: /shop/*English+Bf*
Disallow: /shop/*SEXY-BF*
Disallow: /shop/*sexy%20bf*
Disallow: /shop/*Indian+BF*
Disallow: /shop/*INDIAN+BF*
Disallow: /shop/*indian+bf*
Disallow: /shop/*bf+hot+sexy*
Disallow: /shop/*XXXXX*
Disallow: /shop/*Xxxxx*
Disallow: /shop/*Sexy-Boyfriend-BF*
Disallow: /shop/*Bangladeshi-Bf*
Disallow: /shop/*Bangla-Bf*
Disallow: /shop/*Bangla-Local-Bf*
Disallow: /shop/*English-Bf*

User-agent: Bingbot
Crawl-delay: 1