Robots.txt

Inane baanter or words of wisdom - you decide...
Post Reply
User avatar
sack
Baaaaaaa!
Baaaaaaa!
Posts: 7077
Joined: Fri May 10, 2002 4:13 pm 0
XBL Gamertag: evilgopher
PSN: sixFiveOhTwo
Location: Rijswijk, The Netherlands
Contact:

Re: Robots.txt

Post by sack »

Jedburgh wrote:To be precise, *most* spammers are coming in from gmail addresses. Any other domains get block-banned as a rule. Had a rash of AOLs the other day so that's gone again.

Any and all captcha questions gratefully received.
'Given that god is infinite, and the universe is also infinite... would you like any toast?'
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

disciple wrote:Searches engines are blocked again aren't they?

Any chance we could sort it out? New joins have been gradually dropping since mid 2007 and are now pretty much the lowest they've ever been.

Yakyak's dying a slow death and no-one really seems to give a fuck. :?
Sorry could an admin take a look at this and comment and/or try to resolve the issue please?

It's all a bit frustrating to be honest, this has been round the houses once already. I do feeling like I'm labouring this a bit but no-one else seems particularly bothered about this slow but consistent drop-off since the search engines have been cut off.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

As you know it was turned off after getting denial of service attack on the server.

The -only- thing that had changed on YY in a long while was the removal of robots.txt

So I changed it back and then the denial of service attacks stopped.

Now what would you suggest I do given I don't have a load of time to muck around with this stuff and I really don't want to get DOSsed again as our company website is on the same server
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Here ya go, Yakyak timeline. Joins by month.

Image

Red line is a rough trend indicator, it's pretty flat/consistent till robots goes, then downwards. As mentioned, without search engines there's bugger all way for people to find Yakyak, it strangles the community. The forum is so much quieter than it used to be.

Added the green lines for interest's sake.

Huge surge in join-ups at Space Giraffe release time, just goes to show no publicity is bad publicity. =)

GRR not so much, but at that point we were not being indexed and there's no 2/10 drama going on so no way for the interest to feed back into the forums.

There is a slight recovery in Jan but as well as GRR release it's likely to do with robots being reinstated.

Added XBox release date on just for the Neon thing but it has little bearing on membership presumably visualiser's aren't big news.

I have no idea what the yellow bit that reaches its peak in Feb 2005 is. Can't seem to relate it to anything specific.
Last edited by disciple on Mon Mar 15, 2010 8:21 pm 0, edited 1 time in total.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

So you've no suggestion at all then.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Now what would you suggest I do given I don't have a load of time to muck around with this stuff and I really don't want to get DOSsed again as our company website is on the same server
It's a bit difficult to comment from here, there were quite a few good suggestions put forward last time. from page 3 through to page 7 of this very thread. I have no idea which (if any) of those were implemented.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
Dio
Sheepie Whore
Sheepie Whore
Posts: 18887
Joined: Mon May 31, 2004 1:08 pm 0
Location: Newcastle UK

Re: Robots.txt

Post by Dio »

disciple wrote:I have no idea what the yellow bit that reaches its peak in Feb 2005 is. Can't seem to relate it to anything specific.
Rodent, I would guess - most notably, Paper Arcade and Jeff's History of Llamasoft, both of which broke out into mainstream media (very mainstream in the former), but the readership of Rodent grew very rapidly in that period and at that point the two forums were symbiotic.

IOTW on Rodent is what got me to register here, May 04. Paper Arcade started not long later and built through the back end of 04 - and I'd guess the philosophical split happened sometime in 05 or 06.
Last edited by Dio on Mon Mar 15, 2010 8:39 pm 0, edited 1 time in total.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

gravy wrote:As you know it was turned off after getting denial of service attack on the server.

The -only- thing that had changed on YY in a long while was the removal of robots.txt

So I changed it back and then the denial of service attacks stopped.

Now what would you suggest I do given I don't have a load of time to muck around with this stuff and I really don't want to get DOSsed again as our company website is on the same server
Move it off the same server? We have a Yakyak hosting fund specifically to pay for hosting. If having it on your business server means the community slowly dies then let's take it off.

Regarding the lack of time thing, I suggested this last time but RiK knocked it back saying you have plenty of manpower.

It's either one or the other, not both. Either you have the manpower or you don't. If you do then nominate someone to sort it. If you don't then see if someone on the forum fancies volunteering and/or having a look. We are a tech-savvy forum, I am sure someone here has the skills and inclination and quite possibly experience to manage this.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

I don't have the time to spend on it and have never said otherwise.

If someone else is going to do that sort of admin then it will need to be moved onto another server.

To be honest that's completely fine with me.

I would say that you're conviction that that robots.txt is the issue is just a hunch. I'm really not sure you're right at all.
User avatar
ElfQrin
Baaaaaa!
Baaaaaa!
Posts: 4384
Joined: Mon Jul 28, 2003 12:55 pm 0
Location: Italy
Contact:

Re: Robots.txt

Post by ElfQrin »

I second disciple, for what's worth.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Well maybe. But I guess there's only one way to tell and it's pretty easy to track. Alter robots.txt, get indexed, monitor signups. I struggle to see how else to resolve the "quiet forum" problem if it's not by increasing the active member pool, unless we're planning to mailshot all existing members!
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

ElfQrin wrote:I second disciple, for what's worth.
He probably needs some doers rather than me-tooers at this point
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Dio wrote:
disciple wrote:I have no idea what the yellow bit that reaches its peak in Feb 2005 is. Can't seem to relate it to anything specific.
Rodent, I would guess - most notably, Paper Arcade and Jeff's History of Llamasoft, both of which broke out into mainstream media (very mainstream in the former), but the readership of Rodent grew very rapidly in that period and at that point the two forums were symbiotic.
Yeah that makes sense, I did think it might be Rodent but couldn't figure out how to tell (aka couldn't be arsed checking). The forums were close pre-fallout, can see joins-ups crossing over and having that effect.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gumboot
Baaaaaaa!
Baaaaaaa!
Posts: 7652
Joined: Sun Aug 26, 2007 3:42 am 0
Location: trumpstain!

Re: Robots.txt

Post by gumboot »

gravy wrote:
ElfQrin wrote:I second disciple, for what's worth.
He probably needs some doers rather than me-tooers at this point
Me too.


By the way, I have observed behaviour that could trivially be turned into a DoS against the forum. I would like to report this to the appropriate people. Who are the appropriate people?
User avatar
Flabio
Baaaaaaa!
Baaaaaaa!
Posts: 8717
Joined: Wed Mar 03, 2004 12:37 pm 0
Location: Guildford

Re: Robots.txt

Post by Flabio »

Maybe the reason people dont join here is cause it's basically pretty dull and not a lot really happens anyway? Jeff doesn't post his blog on here anymore and the retro gaming league thing is dead. Almost every gaming thread follows the same pattern (person x posts about game, person y says how game is bad as its not on a Nintendo platform, Meow says its rubbish cause its not 40 hours long, etc...).

The whole place lacks a purpose other than to be somewhere where the locals hang out. It's like a pub in an out of town suburb. If you're not local, there's no reason to go in there even if you know it exists.
Post Reply