PDA

View Full Version : Does robots.txt attract even more spam & unwelcome guests ?



tlwwolfseye
Thu 9th Apr '09, 8:21am
A couple of days ago I put a robots.txt into my Website´s root folder. Now I wonder, since then the number of unwelcome guests and spam seem to increase. Does the fact that one has a robots.txt attract those even more and is maybe not so good to have ?

Thank you in advance.

Wayne Luke
Thu 9th Apr '09, 1:37pm
robots.txt doesn't attract anything. It will allow you to control well-behaved bots. Bots that are up to no good won't even acknowledge that it exists.

People who spam your forums are attracted by the general availability of it. They find them through search engines, social networking and other promotional vehicles. I wouldn't recommend getting off these tools or making your forums unfriendly to search engines. It is something that you will have to deal with. There is no magic pill to make spam disappear. If there was, the hundreds of millions put into this by Google, Microsoft, Yahoo, IBM, Apple and Sun every year would have made it disappear a long time ago.

We work at providing tools that help you deal with this issue but it will always require administrator intervention if you want to keep your site spam-free for the long term.