yahoo?!! please help guys.

[4seasonz]

hi ,

as all the web-masters i submitted my sitemap to google ,
and yahoo before 3 months maybe , i didnt face any problem
with "google" , but i do face a lot of problems with "yahoo spiders"
everyday , i mean the forum now got 4 members online , and 370 yahoo spider,
and the spiders are showing the index of the forum , nothing else



and before couple of days there was about 500 yahoo spider in my forum
not doing anything , just bothering me i guess .

is there anything i can do to reduce "yahoo" spiders in my forum?

[Jose Amaral Rego]

User-Agent: *
Disallow: /
You can find a list to stop spiders from going to certain areas.
http://www.vbulletin.com/forum/showthread.php?t=76662

Googlebot crawls (http://www.google.com/support/webmasters/bin/answer.py?answer=48620)
Yahoobot crawls (http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html)
Msnbot crawls (http://search.msn.com.my/docs/siteowner.aspx?t=SEARCH_WEBMASTER_FAQ_MSNBotIndexi ng.htm&FORM=WFDD#D)


I would just complain to yahoo, as they are just trying to index the heck of every thing right now (Microsoft Takeover)

[setishock]

I only see on the average 15 or 20 yahoo slurp bots at one time. Maybe 2 Google bots and the same for MSN in a 24 hour period.
Maybe it's your subject matter. Care to toss up a link?

[4seasonz]

You can find a list to stop spiders from going to certain areas.

http://www.vbulletin.com/forum/showthread.php?t=76662

Googlebot crawls (http://www.google.com/support/webmasters/bin/answer.py?answer=48620)
Yahoobot crawls (http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html)
Msnbot crawls (http://search.msn.com.my/docs/siteowner.aspx?t=SEARCH_WEBMASTER_FAQ_MSNBotIndexi ng.htm&FORM=WFDD#D)

I would just complain to yahoo, as they are just trying to index the heck of every thing right now (Microsoft Takeover)

thanks ,

but this code or whatever :

User-Agent: *
Disallow: /

disallows all the spiders from indexing my site i guess , including google right?

and i found "set crawl rate " on google , but i didnt find it on yahoo .
any idea where is it?

[4seasonz]

I only see on the average 15 or 20 yahoo slurp bots at one time. Maybe 2 Google bots and the same for MSN in a 24 hour period.
Maybe it's your subject matter. Care to toss up a link?

i see 511 yahoo bots at one time , i dont know why ,
and i see 3-4 google bots at one time , and same for msn
and googlebots and msnbots are normal , but 511 yahoo bot at one time
is not normal.

[Andy]

I only allow Google to index my forum. Here is what my robots.txt looks like.

User-agent: Googlebot
Disallow:

User-agent: *
Disallow: /