Trend Micro crawling my forums?

[JamieFry]

This was mentioned previously but the discussion seemed to veer more towards permissions than why Trend Micro would be trying to crawl my forums. There are currently nine IP addresses registered to Trend Micro trying to access my forums. They can't see anything and get the permissions denied error but how can I make them stop trying?

[Zachery]

I wouldn't worry about it honestly.. its just a spider.

[RossDagley]

This was mentioned previously but the discussion seemed to veer more towards permissions than why Trend Micro would be trying to crawl my forums. There are currently nine IP addresses registered to Trend Micro trying to access my forums. They can't see anything and get the permissions denied error but how can I make them stop trying?

If you areally dont want them to have access (and your forums run on *nix) add their IP to the htaccess file

[alan92rttt]

In this case when their are only 9 IP's crawling your probably right its not a bit issue.

We just fixed a major slowdown caused by spiders. We were running 150-250 users and 200-400 guests. A bit a research revealed that a ton of the guests were from the same IP block and this block resolved to inktomisearch.com We blocked them via a robots.txt and our performance jumped and our guests total dropped to an average of 29.

[feldon23]

I have seen over and over that Yahoo (Inktomi) completely ignores the robots.txt directives that they recommend you use. They swarm over forums with no self-discipline or coordination. Eventually, I banned Yahoo/Inktomi from my website. But some people don't want to be unlisted from Yahoo.

[alan92rttt]

In may case they did completely obey the robots file.

[Andy]

I banned all but Google from my forum.

[alan92rttt]

WHen I was looking intoi the "guests" issue I found one lonely google bot. He can stay he's well behaved.

[JamieFry]

I have robots.txt set to deny all spiders - my forums are completely closed to non-members. Trend Micro is impervious to robots.txt.

I banned the IP block through cPanel. They are gone. I guess I was really just trying to figure out why an anti-virus company would crawl my site. I read somewhere that the new anti-fraud toolbar associated with Trend Micro's latest software is the culprit.

[Floris]

robots.txt is a recommendation for search engines who like to play nice and respect these wishes, ... it isn't a rule set that can prevent access.. That's what .htaccess or firewalls or the tools from cpanel are for

[jonsidneyb]

I wouldn't worry about it honestly.. its just a spider.

I know I am bumping an old thread but these might be a lot more than just spiders.

http://www.projecthoneypot.org/ip_216.104.15.138

They have a lot of comment spam bots it seems.

216.104.15.138
216.104.15.142
216.104.15.130
216.104.15.134
150.70.84.151
150.70.84.24
150.70.84.152

Bad news