Can bots an spriders still roam my forums/posts even if they are set to private and only able to be accessed by registered users????
Can bots an spriders still roam my forums if they are set to private
Collapse
X
-
Can bots an spriders still roam my forums if they are set to private
Tags: None -
As far as I know, spiders and bots have the same permissions as a guest to your forum... so they can only access parts of your forum that a guest can access.
If you check your "who's online" page, you'll see that some spiders are met with the "stop" icon when they try to access parts of your forum that are off-limits to guests, meaning they can only see either a login page or an "access forbidden" message, depending on your forum settings on permissions...
- TOPEN TECH SUPPORT
"Tech is our middle name!" -
yeah i get a lot of those stop signs on my view online members page. i am going to try it without my forums set to private and see what happens over a few days.Comment
-
Why don't you just set up a robots.txt file for your domain root and prevent the spiders from trying to access pages you don't want them to? Part of mine looks like this:
Code:User-agent: * Disallow: /403.html Disallow: /404.html Disallow: /errordocument.html Disallow: /showpic.php Disallow: /community/a-floor-pro-comedy-tour/ Disallow: /community/a-floor-pro-network-support/ Disallow: /community/test-forum/ Disallow: /community/chiefcp/ Disallow: /community/announcement.php Disallow: /community/attachment.php Disallow: /community/calendar.php?do=add Disallow: /community/cron.php Disallow: /community/editpost.php Disallow: /community/joinrequests.php Disallow: /community/login.php Disallow: /community/memberlist.php? Disallow: /community/misc.php Disallow: /community/staffcp/ Disallow: /community/moderator.php Disallow: /community/newreply.php Disallow: /community/newthread.php Disallow: /community/online.php Disallow: /community/payments.php Disallow: /community/poll.php Disallow: /community/postings.php Disallow: /community/printthread.php Disallow: /community/private.php Disallow: /community/profile.php Disallow: /community/register.php Disallow: /community/report.php Disallow: /community/reputation.php Disallow: /community/sendtofriend.php Disallow: /community/search.php Disallow: /community/sendmessage.php Disallow: /community/showgroups.php Disallow: /community/showpost.php Disallow: /community/subscription.php Disallow: /community/subscriptions.php Disallow: /community/threadrate.php Disallow: /community/usercp.php
JimIf my post was helpful to you, please take the time to register at my forum and ask a question you've always wanted to know about floors.
www.TheFloorPro.comComment
-
I was updating my Adsence profile and seen a spot there, you can set it up with a user and pass (I think thats how it is) so it can acccess the content.
But I dont know if the bots that bring in ads are the same as the ones that spider the site.Comment
-
-
If my post was helpful to you, please take the time to register at my forum and ask a question you've always wanted to know about floors.
www.TheFloorPro.comComment
widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
Comment