We had our forum set up for the vb forum Home link to point to site/forums/index.php
We installed Drupal for our CMS in site/ and changed the forum Home Link to point to site.com where Drupal is the home page now.
We DO NOT have vbSEO turned on, we last had it on when we used vb3.x.
On the online list members are getting warning errors when it appears they are trying to access the old Home Link /forums/index.php
How do we turn off the old index page we used for CMS and publishing so members and bots don't see it?
Bing, Yahoo and Baidu appear to be crawling the forum fine but Google seems to have disappeared. Not sure if that is related or not. Google used to index our forum topics almost hourly and now we're 4 pages back if listed at all.
Our robot txt is
Sitemap: site/forums/store_sitemap
User-agent: BoardTracker
Disallow: /
User-agent: SmartViper
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: Twiceler
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YoudaoBot
Disallow: /
User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /
User-agent: NaverBot
User-agent: Yeti
Disallow: /
User-agent: NaverBot
User-agent: Yeti
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: ia_archiver
Allow: /
User-agent: *
Crawl-delay: 10
# Directories
Disallow: /includes/
Disallow: /misc/
Disallow: /modules/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
Disallow: /oscommerce/personal/
Disallow: /oscommerce/includes/
Disallow: /oscommerce/cgi-bin/
Disallow: /oscommerce/banned/
# Files
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /oscommerce/blocked.php
Disallow: /ads/adclick.php
Disallow: *.js
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/index.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/member.php
Disallow: /forums/memberlist.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/payments.php
Disallow: /forums/poll.php
Disallow: /forums/post.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/showpost.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
Disallow: /user/logout/
# Paths (no clean URLs)
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/
We installed Drupal for our CMS in site/ and changed the forum Home Link to point to site.com where Drupal is the home page now.
We DO NOT have vbSEO turned on, we last had it on when we used vb3.x.
On the online list members are getting warning errors when it appears they are trying to access the old Home Link /forums/index.php
How do we turn off the old index page we used for CMS and publishing so members and bots don't see it?
Bing, Yahoo and Baidu appear to be crawling the forum fine but Google seems to have disappeared. Not sure if that is related or not. Google used to index our forum topics almost hourly and now we're 4 pages back if listed at all.
Our robot txt is
Sitemap: site/forums/store_sitemap
User-agent: BoardTracker
Disallow: /
User-agent: SmartViper
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: Twiceler
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YoudaoBot
Disallow: /
User-agent: Baiduspider
User-agent: Baiduspider-video
User-agent: Baiduspider-image
Disallow: /
User-agent: NaverBot
User-agent: Yeti
Disallow: /
User-agent: NaverBot
User-agent: Yeti
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: ia_archiver
Allow: /
User-agent: *
Crawl-delay: 10
# Directories
Disallow: /includes/
Disallow: /misc/
Disallow: /modules/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
Disallow: /oscommerce/personal/
Disallow: /oscommerce/includes/
Disallow: /oscommerce/cgi-bin/
Disallow: /oscommerce/banned/
# Files
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /oscommerce/blocked.php
Disallow: /ads/adclick.php
Disallow: *.js
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/index.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/member.php
Disallow: /forums/memberlist.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/payments.php
Disallow: /forums/poll.php
Disallow: /forums/post.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/showpost.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /node/add/
Disallow: /search/
Disallow: /user/register/
Disallow: /user/password/
Disallow: /user/login/
Disallow: /user/logout/
# Paths (no clean URLs)
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/
Comment