Robots.txt

Inane baanter or words of wisdom - you decide...
Post Reply
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

Flabio wrote:And here comes the spam... hehe.
Yeah :D Just deleted it. Fingers crossed that was just a coincidence. :)
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Good stuff, cheers for sorting that. :)

Regarding spam, no doubt spam's attempts *are* going to increase. But surely there has to be a less severe way of dealing with it than blanket cutting off search engines and slowly throttling the community. I dunno, I don't run a PHPBB but this forum is a great resource! We're not a defenceless knitting forum and I'm sure people here have effectively dealt with spammers before? Fuck I had some unexpectedly marvellous advice from D_Dave about securing my web server the other week.
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
Flabio
Baaaaaaa!
Baaaaaaa!
Posts: 8717
Joined: Wed Mar 03, 2004 12:37 pm 0
Location: Guildford

Re: Robots.txt

Post by Flabio »

We're not as 'active' with out moderation though. The people that run this place do it for the fun of it for the most part, so we don't always have someone online to tidy up when spam strikes.

But yeah, it has gone very quiet here lately.
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

Personally I'm don't mind doing stuff but I don't want to spend time mucking about trying things, I don't have the time.

Totally up for trying specific approaches that it's clear how to implement them that someone else has done the legwork researching
Flabio wrote:We're not as 'active' with out moderation though. The people that run this place do it for the fun of it for the most part, so we don't always have someone online to tidy up when spam strikes.
There's quite a lot of it right now, 16 in the last 3 days. You only really see the ones that slip through but the majority get dealt with very quickly.

It's just if the amount grows the workload would be a pain.

Let's see what happens
User avatar
gumboot
Baaaaaaa!
Baaaaaaa!
Posts: 7652
Joined: Sun Aug 26, 2007 3:42 am 0
Location: trumpstain!

Re: Robots.txt

Post by gumboot »

disciple wrote:With the "attracting the wrong sort" thing you could control this with good use of robots.txt, enable it for all forum sections EXCEPT bleatings, that way the only stuff that gets indexed is the gaming, musician, programming and similar stuff, i.e. the core Yakyak content.
Also, can't this be done outside of robots.txt, in the headers for each page that shouldn't be indexed?
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

Flabio wrote:We're not as 'active' with out moderation though. The people that run this place do it for the fun of it for the most part, so we don't always have someone online to tidy up when spam strikes.
If this is an issue I'm sure there's plenty of people here who'd assist if it was helpful. The community is pretty much self supporting, my understanding is the donations more than pay for hosting, if more moderators or administrators are needed then FFS let's add some.

The same with gumboot's feature request, there's no reason that this couldn't be added with a plugin, all that would be needed is for someone to write one and agree to keep it updated should we move to PHP 4. It really doesn't have to be "1 person doing all the work, so we can't do much because that one person doesn't have the time", I'm sure there are plenty of will and capable people here who've been around long enough to be trusted not to wipe the database at first opportunity!
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

I'm a big fan of the people asking for work to be done do the work.

There's absolutely nothing stopping anyone installing their own phpbb3 board on there own machine and do feature updates. I'd be happy to install them
User avatar
RiK
Sheepie Whore
Sheepie Whore
Posts: 16092
Joined: Tue Apr 23, 2002 11:32 pm 0
XBL Gamertag: Dr Olpin
PSN: Dr Thumpy
Location: The Shire
Contact:

Re: Robots.txt

Post by RiK »

disciple wrote:If this is an issue I'm sure there's plenty of people here who'd assist if it was helpful. The community is pretty much self supporting, my understanding is the donations more than pay for hosting, if more moderators or administrators are needed then FFS let's add some.
There are more than enough already.
disciple wrote:The same with gumboot's feature request, there's no reason that this couldn't be added with a plugin, all that would be needed is for someone to write one and agree to keep it updated should we move to PHP 4. It really doesn't have to be "1 person doing all the work, so we can't do much because that one person doesn't have the time", I'm sure there are plenty of will and capable people here who've been around long enough to be trusted not to wipe the database at first opportunity!
The simple fact is, maintaining a customised install is a significant amount of work compared to running a standard board. Adding more people doesn't help as it would still fundamentally fall back to whoever did the mod in the first place to have sole responsibility for it.
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

RiK wrote: The simple fact is, maintaining a customised install is a significant amount of work compared to running a standard board. Adding more people doesn't help as it would still fundamentally fall back to whoever did the mod in the first place to have sole responsibility for it.
To be fair a plugin wouldn't have as much trouble, there's a pretty good architecture in phpbb3 to add new plugins that can operate across board updates.

That said I'd be a bit wary of installing a plugin that's been written by someone who doesn't have a track record making them.

As I said if people feel this stuff is important there's absolutely no reason why they can't get stuck on their own private install as a proof of concept and then propose it's something we should use here.
User avatar
gumboot
Baaaaaaa!
Baaaaaaa!
Posts: 7652
Joined: Sun Aug 26, 2007 3:42 am 0
Location: trumpstain!

Re: Robots.txt

Post by gumboot »

disciple wrote:The same with gumboot's feature request,
It was just an implementation suggestion, just in case that sort of thing fits better into the structure of the thing. I'd have thought it would, because page layout is a pretty obvious customisation; unless you can only do that through style sheets, and I don't imagine robots even bother with those, let alone support a keep-out flag in there.

I don't know anything about webdev, so I don't know if it's easier or harder than any other way. My own inclination would be to put a proper hierarchical directory structure on the mainteners' TODO list and walk away.

Otherwise, unless you want the board moved to an embedded processor, rewritten in assembly, and SIMD optimised, I probably can't help.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

gravy wrote:To be fair a plugin wouldn't have as much trouble, there's a pretty good architecture in phpbb3 to add new plugins that can operate across board updates.

That said I'd be a bit wary of installing a plugin that's been written by someone who doesn't have a track record making them.

As I said if people feel this stuff is important there's absolutely no reason why they can't get stuck on their own private install as a proof of concept and then propose it's something we should use here.
Sounds great, might have a muck around at home and see what the state of plugins for embedding more than just youTube media is like. Assuming changelogs get updated with version upgrades, we're 3.01 right? http://yakyak.org/docs/CHANGELOG.html
PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

Have a read about how it works before making assumptions how you do things or how stuff works or what you would do to include a certain feature.

For example you can already embed flash without the need for plugins, it's a pretty minor thing to do.
User avatar
disciple
Sheepie Whore
Sheepie Whore
Posts: 13941
Joined: Mon Oct 27, 2003 11:33 pm 0
XBL Gamertag: LiberMentis
PSN: Prime_Heretic
Location: Greenfield
Contact:

Re: Robots.txt

Post by disciple »

PS4 : Prime_Heretic
Steam: bitstorm
Elite Dangerous: Cmdr Britain
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Robots.txt

Post by gravy »

And we've had a marked increase in spammers.

Great!
User avatar
ElfQrin
Baaaaaa!
Baaaaaa!
Posts: 4384
Joined: Mon Jul 28, 2003 12:55 pm 0
Location: Italy
Contact:

Re: Robots.txt

Post by ElfQrin »

you can't have a rose without the thorn.
Post Reply