how to - google not indexing certain areas of forum

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • numberone
    Member
    • Feb 2005
    • 61
    #1

    how to - google not indexing certain areas of forum

    How do I keep google from indexing meaningless pages such as http://www.mysite.com/forum/calendar.php?do=add&c=1&day=2003-10-16?

    Any other good references that on what google does not really need to be indexing and how to keep it from doing so?

    Thanks
    Tags: None
    • furst
      Senior Member
      • Dec 2005
      • 202
      • 3.5.x
      #2
      Create a robots.txt file and add the exclusion.

      The Web Robots Pages
      http://www.robotstxt.org/


      Here's a good list of locations to exclude:
      Code:
      [COLOR=#000000][COLOR=#0000bb]User[/COLOR][COLOR=#007700]-[/COLOR][COLOR=#0000bb]agent[/COLOR][COLOR=#007700]: *
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]admincp[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]cgi[/COLOR][COLOR=#007700]-[/COLOR][COLOR=#0000bb]bin[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]clientscript[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]includes[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]install[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]modcp[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]sitemap[/COLOR][COLOR=#007700]/
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]tags[/COLOR][COLOR=#007700]/

[/COLOR][COLOR=#0000bb]User[/COLOR][COLOR=#007700]-[/COLOR][COLOR=#0000bb]agent[/COLOR][COLOR=#007700]: *
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]printthread[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]subscription[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]payments[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]profile[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]faq[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]calendar[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]private[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]sendmessage[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]sendmessage[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php[/COLOR][COLOR=#007700]?do=
[/COLOR][COLOR=#0000bb]Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]showgroups[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]reputation[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]report[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]threadrate[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]postings[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]newthread[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]search[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]newreply[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]register[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]login[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]faq[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]image[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]cron[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]joinrequests[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php
Disallow[/COLOR][COLOR=#007700]: /[/COLOR][COLOR=#0000bb]forums[/COLOR][COLOR=#007700]/[/COLOR][COLOR=#0000bb]usercp[/COLOR][COLOR=#007700].[/COLOR][COLOR=#0000bb]php[/COLOR][/COLOR]

        Comment

        Previous template Next
        widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
        Working...