Robots.txt

Robots.txt Rules

Crawler Rules
mediapartners-google*
  • Allow: /
  • Disallow:
Yahoo!
  • Allow: /
adsbot-google*
  • Allow: /
  • Disallow:
All
  • Allow: /
  • Disallow: /*.pdf$
  • Disallow: /*.txt$
  • Disallow: /*.html$
  • Disallow: /file/
  • Disallow: /howto/
  • Disallow: /image/

Sitemaps

Priority Location
Nothing to show.

Raw Text

User-agent: Mediapartners-Google*
Disallow:

User-agent: Slurp
Crawl-delay: 1

User-agent: AdsBot-Google*
Disallow:

User-agent: *
Disallow: /*.pdf$
Disallow: /*.txt$
Disallow: /*.html$
Disallow: /file/*
Disallow: /howto/*
Disallow: /image/*