Robots.txt - disallow

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • okc
    Senior Member
    • Oct 2004
    • 235

    Robots.txt - disallow

    I am not sure why I want to disallow bots from searching through my site with the "disallow: / " command - don't I want them to spider everything to help my rankings?

    thx
  • Jose Amaral Rego
    Senior Member
    • Feb 2005
    • 11058
    • 1.1.x

    #2
    Ah Danielson, it is an age old question. Humm, you should answer this question yourself and fight off the demons from within. Yes, that is what is to come. Now go forth and learn what wisdom you gain.

    Comment

    • okc
      Senior Member
      • Oct 2004
      • 235

      #3
      Are you speaking in tongues or was that a cryptic response to me question

      Comment

      • Jose Amaral Rego
        Senior Member
        • Feb 2005
        • 11058
        • 1.1.x

        #4
        Originally posted by okc
        Are you speaking in tongues or was that a cryptic response to me question
        A reason to keep out spiders would be to save bandwidth or you do not want certain info google all over the net.

        You can stop bot's then let them in so they can place new post up then shut them out every few weeks or whatever time frame you have.

        It all depends on what you want. If it a game by who's ranked the best then let them in all the time or if it is bandwidth then don't

        Comment

        widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
        Working...