Robots.txt

[disciple]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

[piku]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

Replace your name with Gravy's or anyone else who's posted in here and it works too.

[Harlequin]

lol
http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

Replace your name with Gravy's or anyone else who's posted in here and it works too.

disciple and yourself get one result each; gravy gets two!

[craigt]

This explains it...

http://www.sitepoint.com/blogs/2009/12/ ... in-google/

Another website somewhere must have linked to this thread, mentioning desciple in the process. Are all our ears burning?

[Alarm]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

Replace your name with Gravy's or anyone else who's posted in here and it works too.

Mine doesn't (at least not for the first two pages of results).

[PVB]

Feel like I'm being stalked. How did that get in there then, manually submitted?

Replace your name with Gravy's or anyone else who's posted in here and it works too.

Mine doesn't (at least not for the first two pages of results).

That's because Google's ignoring you.

Edit: It's ignoring me too.

[Baggypants]

http://uk.search.yahoo.com/search?p=bag ... 8&fr=moz35

[gravy]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

As it's been pointed out loads of times a lot of the site is currently indexed on google. Yet new sign ups still decline. I think that's the strongest indication yet that robots.txt is irrelevant

[Harlequin]

As it's been pointed out loads of times a lot of the site is currently indexed on google. Yet new sign ups still decline. I think that's the strongest indication yet that robots.txt is irrelevant

Curious. Which bits are and are not indexed?

[gravy]

As it's been pointed out loads of times a lot of the site is currently indexed on google. Yet new sign ups still decline. I think that's the strongest indication yet that robots.txt is irrelevant

Curious. Which bits are and are not indexed?

No idea. As I said earlier in the thread what's in google is a hangover from the time recently when spidering was allowed.

[tufty]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

As it's been pointed out loads of times a lot of the site is currently indexed on google. Yet new sign ups still decline. I think that's the strongest indication yet that robots.txt is irrelevant

It places extremely poorly for the type of queries that might pull a punter in, though: "Space Giraffe", "Gridrunner", or even "Jeff Minter" don't get a hit in the first page, and even "llamasoft" only gets this "helpful" link, at hit 8:

[gravy]

I think that's more to do with the fact that Space Giraffe, Jeff Minter and GRR just aren't a big part of the site. I suspect that Llamasoft gets a hit because it's in the forum banner at the top. More indexing isn't going to change the state of affairs, if you want those terms more closely linked to this site by Google then you're going to need to do something else.

It sounds like you're after some sort of <shudder> SOE

[tufty]

I think that's more to do with the fact that Space Giraffe, Jeff Minter and GRR just aren't a big part of the site. I suspect that Llamasoft gets a hit because it's in the forum banner at the top. More indexing isn't going to change the state of affairs, if you want those terms more closely linked to this site by Google then you're going to need to do something else.

It sounds like you're after some sort of <shudder> SOE

I dunno. ISTR when I joined, this site was the second hit for "gridrunner" (google was how I got here), and had something more than just "www.yakyak.org" as the description.

The "no description" thing is, as far as I can assess, a direct result of google not being able to spider.

[craigt]

I think that's more to do with the fact that Space Giraffe, Jeff Minter and GRR just aren't a big part of the site. I suspect that Llamasoft gets a hit because it's in the forum banner at the top. More indexing isn't going to change the state of affairs, if you want those terms more closely linked to this site by Google then you're going to need to do something else.

It sounds like you're after some sort of <shudder> SOE

That's not the case. This is easily shown by searching for "space giraffe forum". A lot of forums appear which only have one thread and a couple of posts of Space Giraffe discussion. Yakyak doesn't appear even though it has much more content on the subject.

As explained in the link I gave a few posts ago, Google is only indexing pages of the forum which have been linked to from other pages on the web. In your book, this gives us the worst of both worlds... The forum does appear in Google results (which defeats the purpose of turning off robots.txt), but any links which do appear are badly indexed.

So... If the main reason to turn off robots.txt was to stop people finding the site to prevent DDoS, but Google is still indexing the site (badly), why can't robots.txt be turned back on again?

<hides behind sofa>

[gravy]

lol

http://www.google.co.uk/search?hl=en&so ... =&gs_rfai=

Feel like I'm being stalked. How did that get in there then, manually submitted?

And someone's just manually removed this search result from google!

There's a lot of silliness going one here

So... If the main reason to turn off robots.txt was to stop people finding the site to prevent DDoS, but Google is still indexing the site (badly), why can't robots.txt be turned back on again?

<hides behind sofa>

Robots.txt doesn't just disallow google, it disallows a lot of other indexing. But yeah you're probably right.

But this has brought up a bigger issue for me. The forum from a tech point of view is now very very stable, it takes up sod all admin time. Then I get badgered into making one change and all of a sudden we had issues though none of the benefits the change was supposed to bring. I swapped that change back and there's no issues which to be honest that is how I like it.

So I don't mind hosting this site as long as there's nothing to do and I don't think it unreasonable that I have some say so in that as it's my time What I do mind is people with no experience or understanding of the issues basically making up worklists for me to do. Particularly when the only argument is constantly repeating a proven fallacy.

So rather than moan about that it I'm happy to move the site off and let someone else get JUNIOR ADMIN SQUAD in their ear