Robots.txt

Inane baanter or words of wisdom - you decide...
Post Reply
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

I think that's the biggest problem I have with JUNIOR ADMIN SQUAD, full of advice about work the mods or admins should do but a flurry of reasons not to do anything themselves.

- Disk space, you'd probably need a couple of gig at least
- The server will need to support PHP and mysql
- I'd advise you get an account where you can exercise tight control on the myslq server; this was the problem with blackcat, it just wasn't tuned to run quickly for YY but you weren't able to admin it yourself
- You need to make sure you backup the database (I do a four day rolling back up on the current one)
- Not sure about bandwidth usage to be honest
- You need shell acount access
- It could work with on a WINDERS server but least hassle is another Linux host
- I don't know about the BC costs but there's a thread about it on here somewhere with the info you need

You've got more than enough info there to put together a proposal for what you want to do.

Personally I really don't mind as once it's off my server it's not my problem
adam
Baaaaaa!
Baaaaaa!
Posts: 4418
Joined: Sat Jun 24, 2006 9:15 pm 0

Re: Robots.txt

Post by adam »

speaking of sql...

Image
User avatar
gumboot
Baaaaaaa!
Baaaaaaa!
Posts: 7652
Joined: Sun Aug 26, 2007 3:42 am 0
Location: trumpstain!

Re: Robots.txt

Post by gumboot »

gilesgoat wrote:The final message is "do not even assume not even for a minute that data in the internet is secure".
FTFY.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Well the rough plan was :

Lunix VM hosting.
Mysql - no brainer, I would prefer to use Postgres but it's no-where near as widely used with PHPBB and although PHPBB reckons it supports it I'm not so sure. Not 100% on backend choice, probably MyISAM though InnoDB might sort the deletion locking the forum issue, I dunno at this point, not sure how much slower InnoDB would over MyISAM. Something to explore I think.
PHP - no brainer.
Nginx frontend - lightweight front end running PHPBB through Fastcgi.
Backups - not thought about this too much, I know one of my colleagues runs a PHPBB forum and emails MySQL backups to himself daily. Nice and neat and does the job. Would have to look a bit closer at this, it might be ok for diffs but the base backup I presume would be far too huge to make this workable on Yakyak. I have webhosting already with Linode, I seem to remember that inter Linode data transfer is pretty cost effective. Maybe Amazon. Would have to look more into this.
Upgrades - I'm also not 100% sure at this point on what the best way is to manage the install. It would be nice to have it all managed, security patched and upgraded through distro packages but there is the issue of plugins. I understand PHPBB 2.0 plugins were added with manual changes to the code meaning any upgrade would nuke all plugins. I believe PHPBB3 handles it much better, would need to research all this a bit more.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gilesgoat
Sheepie Whore
Sheepie Whore
Posts: 27258
Joined: Wed Sep 03, 2003 7:54 am 0
Location: Monfalcone, Italy
Contact:

Re: Robots.txt

Post by gilesgoat »

Just a sec help me to re-understand this ..

So all this to basically re-enable robots.txt to index this forum right ?

And the theory behind all this is that this forum is not getting enough NEW registrations cause we are not so easy to be
found on the internet so restoring it should boost things up.

However, if that is true this also mean that the amount of spam/crap/trolls will boost up as well, horse sense would say that
probably both things ( spam and registration ) could be in fact going to be boosted at least a little after that goes on.

I think it's going to be, at least initally, quite a bit of "admin work" to increase one effect without increasing the other as well.

I think it's going to be quite a tought job, qdos to whoever is going to do it because I am sure it's going to be more
problematic than what it looks.
Giles the billy goat
"Baa"
User avatar
ElfQrin
Baaaaaa!
Baaaaaa!
Posts: 4384
Joined: Mon Jul 28, 2003 12:55 pm 0
Location: Italy
Contact:

Re: Robots.txt

Post by ElfQrin »

gilesgoat wrote: However, if that is true this also mean that the amount of spam/crap/trolls will boost up as well, horse sense would say that
probably both things ( spam and registration ) could be in fact going to be boosted at least a little after that goes on.

I think it's going to be, at least initally, quite a bit of "admin work" to increase one effect without increasing the other as well.

I think it's going to be quite a tought job, qdos to whoever is going to do it because I am sure it's going to be more
problematic than what it looks.
I'm in fact thinking to shut down the Internet.
User avatar
Harlequin
Baaaaaaa!
Baaaaaaa!
Posts: 6505
Joined: Wed Jun 05, 2002 4:19 am 0
Location: London, directly above the centre of the Earth.

Re: Robots.txt

Post by Harlequin »

disciple wrote:
Jamesac wrote:My solution- collapse a few sub-forums back into Bleatings. But nobody asked, and it's a suggestion that's been dismissed before. Still, if it were up to me (which it's never going to be) I'd chop Ooh Shiny and Dancing About Architecture, despite them being two of the best-named sub-forums ever.
How do you think this would help sort the issue?
I personally suspect that the vast majority of members, like myself, do not routinely check the non-Bleatings sub-forums nor use 'View unread/new posts' (which appears to defeat the purpose of having sub-forums in the first place). I was discussing this with someone recently in relation to the photo competition along with the confusion of some television programme threads being in Bleatings and others in DAA.
Emoticon and contraction free since... quite a while ago.
User avatar
gribbles
Baaaaaa!
Baaaaaa!
Posts: 3882
Joined: Tue Apr 16, 2002 5:51 pm 0
XBL Gamertag: gribs
PSN: gribbles-

Re: Robots.txt

Post by gribbles »

google: llamasoft
1st result - www.llamasoft.co.uk - yakyak linked to off menu on front page and in about.

google: yakyak
1st result - www.yakyak.org

google: gridrunner
1st result - http://www.llamasoft.co.uk/gridrunner.php - yakyak linked to off menu on page

google: space giraffe
2nd result - www.llamasoft.co.uk - yakyak linked to off menu on front page and in about.

google: minter
2nd result - www.llamasoft.co.uk - yakyak linked to off menu on front page and in about

Not difficult to find, really.

To me there's another question. How many people are visiting, and how many of those are registering and posting? It's possibly a conversion issue, not number of visitors.
User avatar
Jamesac
Baaaaaaa!
Baaaaaaa!
Posts: 6677
Joined: Thu Jun 19, 2003 7:23 am 0
Location: Maine
Contact:

Re: Robots.txt

Post by Jamesac »

Yep. I don't bookmark my non-work-related sites at work, and instead go for short Google search strings. Yakyak is all I need to bring this place up first thing.

And Harlequin basically nails it for why I'd condense those two sub-forums back into Bleatings. It would add a bit more focus to the forums overall, and make for a livelier Bleatings.

James
User avatar
gumboot
Baaaaaaa!
Baaaaaaa!
Posts: 7652
Joined: Sun Aug 26, 2007 3:42 am 0
Location: trumpstain!

Re: Robots.txt

Post by gumboot »

gribbles wrote:google: llamasoft
1st result - http://www.llamasoft.co.uk - yakyak linked to off menu on front page and in about.
No such luck with "mutant camels" or "iridis alpha". "Llamatron" and "Lightsynth" fare better, though.

Also, no luck with "cretins".
User avatar
RiK
Sheepie Whore
Sheepie Whore
Posts: 16092
Joined: Tue Apr 23, 2002 11:32 pm 0
XBL Gamertag: Dr Olpin
PSN: Dr Thumpy
Location: The Shire
Contact:

Re: Robots.txt

Post by RiK »

Harlequin wrote:I personally suspect that the vast majority of members, like myself, do not routinely check the non-Bleatings sub-forums nor use 'View unread/new posts' (which appears to defeat the purpose of having sub-forums in the first place). I was discussing this with someone recently in relation to the photo competition along with the confusion of some television programme threads being in Bleatings and others in DAA.
I really can't understand the fuss about forum sections and sub-forums. Some people seem to moan about them all the time so what to they do on their home PCs then, just stick everything in one big folder and hope they can find it again? If they went to the library would they like all the books just stacked randomly regardless of genre or perhaps in the shops they'd like to just pile everything up in one big pile to sort through?

It's called a filing system, it makes things you're interested in easier to find. I hear they're all the rage these days.

As for the issue of threads in the wrong place, personally I always do my general browsing via 'view active topics' so don't necessarily spot threads in the wrong section. if you happen to spot one then please let a mod know and we can easily move it to the correct section.
User avatar
RiK
Sheepie Whore
Sheepie Whore
Posts: 16092
Joined: Tue Apr 23, 2002 11:32 pm 0
XBL Gamertag: Dr Olpin
PSN: Dr Thumpy
Location: The Shire
Contact:

Re: Robots.txt

Post by RiK »

Jamesac wrote:And Harlequin basically nails it for why I'd condense those two sub-forums back into Bleatings. It would add a bit more focus to the forums overall, and make for a livelier Bleatings.
Why does bleatings *have* to be 'livelier', when essentially it's just the 'off-topic' random chat thread. Sticking topic-specific threads in their doesn't improve it, it just means it has more threads which are better suited in their own sections for the people who are interested in those specific topics.
User avatar
Harlequin
Baaaaaaa!
Baaaaaaa!
Posts: 6505
Joined: Wed Jun 05, 2002 4:19 am 0
Location: London, directly above the centre of the Earth.

Re: Robots.txt

Post by Harlequin »

RiK wrote:I really can't understand the fuss about forum sections and sub-forums. Some people seem to moan about them all the time so what to they do on their home PCs then, just stick everything in one big folder and hope they can find it again? If they went to the library would they like all the books just stacked randomly regardless of genre or perhaps in the shops they'd like to just pile everything up in one big pile to sort through?
It's called a filing system, it makes things you're interested in easier to find. I hear they're all the rage these days.
As for the issue of threads in the wrong place, personally I always do my general browsing via 'view active topics' so don't necessarily spot threads in the wrong section. …
So you do your general browsing as if everything was in "one big folder", "stacked randomly" in "one big pile"?
RiK wrote:… if you happen to spot one then please let a mod know and we can easily move it to the correct section.
I shall decline, thanks all the same. I do not want them moved out of Bleatings either.

*Edit*
I suspect that there might be some discrepancy between how you believe the forum should be navigated and how most people actually use it.
Emoticon and contraction free since... quite a while ago.
User avatar
craigt
Baaaa!
Baaaa!
Posts: 896
Joined: Tue Feb 15, 2005 12:23 am 0
Location: Ye Olde Somersetshire

Re: Robots.txt

Post by craigt »

gumboot wrote:
gribbles wrote:google: llamasoft
1st result - http://www.llamasoft.co.uk - yakyak linked to off menu on front page and in about.
No such luck with "mutant camels" or "iridis alpha". "Llamatron" and "Lightsynth" fare better, though.

Also, no luck with "cretins".
No mention of yakyak on a search for "space giraffe" or "Gridrunner Revolution" either. Presumably because they were released after search indexing was diabled?

One way to get more activity on the forum would be to integate the Llamasoft support forum into yakyak, or move yakyak to the support forum. I never really understood why they are seperate (but i'm sure someone will tell me ;) ). People are more likely to track down the community if they have a problem that needs sorting out.
User avatar
RiK
Sheepie Whore
Sheepie Whore
Posts: 16092
Joined: Tue Apr 23, 2002 11:32 pm 0
XBL Gamertag: Dr Olpin
PSN: Dr Thumpy
Location: The Shire
Contact:

Re: Robots.txt

Post by RiK »

Harlequin wrote:So you do your general browsing as if everything was in "one big folder", "stacked randomly" in "one big pile"?
That's where I keep a general idea on what's going on around the whole forum yes. As an admin that's necessary.

When I wish to look at specific things which interest me, for example the programming or photo com threads then I hit the board index link and go to that specific section.
Harlequin wrote:I suspect that there might be some discrepancy between how you believe the forum should be navigated and how most people actually use it.
No, I simply feel that given that every forum I've ever visited has many topic-related sections, and that they are analogous to ever other computer-based filing system which we are all very familiar with then i cannot see any logical reason not to use them. Especially when features such as 'view new posts' and 'view active topics' allow the 'we want it all on one place' crowd to use the forum that way if they wish whilst retaining the topic-specific sections for the people who prefer a rather more organised approach. As it stands the features exist to use it either way.

And before someone jumps in with'but view new posts doesn't work for me as I only visit once a month' then I respectfully suggest that yes, forums features are geared towards the more frequent visitors and quite rightly too as surely they should take precedence over people who only browse casually on an sporadic basis.
Post Reply