Getting Google to spider my board

[Ian Montgomerie]

So, I have VBulletin (currently at 3.01) and Google has never spidered my board beyond the top few pages. In particular it never goes to the archive, even though I can get to the archive just fine as a guest, and the board has been running for five months.

Googlebot visits very regularly but doesn't seem to go much beyond the root page. However MSNBot and Inktomi Slurp do index my archive. I submitted my site to Google both as the root URL, and as the archive URL. Previous to using VBulletin I had a Discus board. Google did spider about 100-200 of the Discus board's messages, but that's out of 60,000.

Any suggestions of what might be going on?

Site is:

http://www.alternatehistory.com/discussion (http://www.alternatehistory.com/discussion)

[Zachery]

Have you ever submitted your board to google?

Can guests view all your pages? if they cant, google wont and cannot spider much.

[Ian Montgomerie]

As I said, it's guest-viewable (I confirmed by clearing my cookies), and Google does know about the site (I submitted it, and it regularly hits the forumhome and a few other pages, but it's never gone into the archive or the board threads themselves).



I'm wondering if there could be some Google-specific reason, because MSNBot and Inktomi Slurp do seem to be spidering the archive slowly but surely.