Robots.txt

[Alarm]

Has anyone got a copy of Robots.pdf? I haven't got a text viewer to hand.

[tufty]

Has anyone got a copy of Robots.pdf? I haven't got a text viewer to hand.

http://www.datafilehost.com/get.php?file=68116881

[Alarm]

Has anyone got a copy of Robots.pdf? I haven't got a text viewer to hand.

http://www.datafilehost.com/get.php?file=68116881

Accessing directly the download link doesn't work. The download only starts if you click from the download page.

[tufty]

Has anyone got a copy of Robots.pdf? I haven't got a text viewer to hand.

http://www.datafilehost.com/get.php?file=68116881

Accessing directly the download link doesn't work. The download only starts if you click from the download page.

Oh, bollocks.

Try this instead : http://tinyurl.com/2q7pp

[Alarm]

Thanks, that works a treat.

[Baggypants]

Is there a Robots.xls?

[disciple]

A stealth change!

User-agent: *
Disallow: /post-*.html$
Disallow: /updates-topic.html*$
Disallow: /stop-updates-topic.html*$
Disallow: /ptopic*.html$
Disallow: /ntopic*.html$
Disallow: /admin/
Disallow: /db/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /templates/
Disallow: /common.php
Disallow: /groupcp.php
Disallow: /memberlist.php
Disallow: /modcp.php
Disallow: /posting.php
Disallow: /profile.php
Disallow: /profile.php?mode=register
Disallow: /privmsg.php
Disallow: /viewonline.php
Disallow: /faq.php
Disallow: /search.php
Disallow: /search.php?search_id=unanswered
Disallow: /login.php

Good job!