[RELEASE] vbSpiderFriend - Search Engine Friendliness

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Overgrow
    Senior Member
    • May 2000
    • 381

    [RELEASE] vbSpiderFriend - Search Engine Friendliness

    I am tired of my 200,000 posts not being listed in Google. I was inspired by phpbuilder.com this morning and I wrote:

    vbSpiderFriend - the search engine indexer for all of your posts

    Purpose: Allow search engine spiders to crawl a linked list of all of your posts.

    Project Requirements:

    -Friendly URLs (no query strings)
    -Good dynamic meta tags
    -Never have to touch the script again.. It is Y3K compliant, simply re-submit to the engines to update your listings

    Install Requirements:

    -vBulletin 1.x or 2.x
    -about 10 minutes


    1) Download the attached Zip.

    2) Open class.mysql.php and put your database login info at the top.

    3) Create a new directory called archive under your forum, like /forum/archive

    4) Open the included .htaccess and change the Error 404 to your new archive path.

    5) Open index.php and change the self-explanatory variables at the top of the file.

    6) Upload all 3 files to your archive directory.

    7) Submit /forum/archive/index.php to search engines and watch em crawl


    DISCLAIMER: I don't use 2.x but I checked the schema and this should work fine.

    NOTES: This uses ErrorDocument and query string parsing to get the variables needed. I do not have the time or energy to troubleshoot this if it does not work on your server. Sorry!
    vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
    Overgrow the Government: The Edge Forums
    presented by Overgrow Marijuana Magazine


    *warning: adult content since vB1.1... now faster than ever!
  • Overgrow
    Senior Member
    • May 2000
    • 381

    #2
    FILE ATTACHED

    ...

    VERSION 1.0

    NEW VERSION, May 21 9:38 am PST


    Upgrading: Unzip and copy your variables from the top of the old index.php to the new index.php.


    FIXES:

    v1.1b release

    -That forwarding scheme added in 1.0b is considered 'cloaking' by Google so it has been turned off by default. If you wish to enable it, make $refresh=1 in the top options.

    -Added a new link at the top saying that this is the text-only version, click for the real thing. Idea by robertusss.

    v1.0b release:

    -Added a forwarding scheme. If the archive is spidered they will see the search-friendly version of the thread. If a user clicks onto that page from a search engine, they will be automatically forwarded to the real showthread.php. This is done with the REFERER tag.

    -Added the top link to the footer as well for more keyword density

    -Fixed all minor bugs

    v0.1a:

    -Made the private forums actually hide themselves

    -Made the query string parser more flexible to work on any install location

    -Added $privateForums variable so they will not be shown

    -Added header("Status: 200 OK"); for the really picky engines
    Last edited by Overgrow; Mon 21 May '01, 8:32am.
    vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
    Overgrow the Government: The Edge Forums
    presented by Overgrow Marijuana Magazine


    *warning: adult content since vB1.1... now faster than ever!

    Comment

    • Overgrow
      Senior Member
      • May 2000
      • 381

      #3
      View an online example:

      vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
      Overgrow the Government: The Edge Forums
      presented by Overgrow Marijuana Magazine


      *warning: adult content since vB1.1... now faster than ever!

      Comment

      • George L
        Former vBulletin Support
        • May 2000
        • 32996
        • 3.8.x

        #4
        woah nicely laid out output too
        :: Always Back Up Forum Database + Attachments BEFORE upgrading !
        :: Nginx SPDY SSL - World Flags Demo [video results]
        :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

        Comment

        • Overgrow
          Senior Member
          • May 2000
          • 381

          #5
          I know you must be piling on the sarcasm.. cracking on my UI like that! Look how flexible it is-- you can change the spacing to however many nbsp's you want! hehe

          This is never meant to be viewed by a user except when they click through the search engine to the plain-text thread. Then the point is to get them to click on one of the top links to bring them into your real forum system. Before I launch this on my site I am definitely going to "pretty it up" but I figured I'd release the raw code and let everyone else have at it as well.

          p.s. my site is damn slow today, I'm sure when installed on a real system it won't be so slow as that example.
          vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
          Overgrow the Government: The Edge Forums
          presented by Overgrow Marijuana Magazine


          *warning: adult content since vB1.1... now faster than ever!

          Comment

          • George L
            Former vBulletin Support
            • May 2000
            • 32996
            • 3.8.x

            #6
            strange i just installed this on my vb 2 rc2 forum and it loads with all forums but no threads are shown when i click on any of the forum links/dates ?

            i'd post the url but my private forums are revealed as well

            is this meant to be set to mysql

            var $CONN = "";
            Last edited by George L; Mon 30 Apr '01, 3:38pm.
            :: Always Back Up Forum Database + Attachments BEFORE upgrading !
            :: Nginx SPDY SSL - World Flags Demo [video results]
            :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

            Comment

            • Overgrow
              Senior Member
              • May 2000
              • 381

              #7
              Does it tell you "No posts, please go back" ?

              Does it give you a totally blank screen?

              or does it at least show what forum you are in?

              This change is just for eva to troubleshoot... FIND

              Code:
              echo spacer(2)."Dates: $date1 to $date2<br>&nbsp;<br>";
              
              	$query = "SELECT title,threadid,lastpost FROM thread 
              WHERE lastpost > '$ts1' AND lastpost < '$ts2' 
              AND forumid='$forumID' ORDER BY dateline ASC";
              change to

              Code:
              echo spacer(2)."Dates: $date1 to $date2<br>&nbsp;<br>";
              
              	$query = "SELECT title,threadid,lastpost FROM thread 
              WHERE lastpost > '$ts1' AND lastpost < '$ts2' 
              AND forumid='$forumID' ORDER BY dateline ASC";
              
              echo $query;
              Then when it spits the query out, run it in phpmyadmin and see if it is a valid query.
              vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
              Overgrow the Government: The Edge Forums
              presented by Overgrow Marijuana Magazine


              *warning: adult content since vB1.1... now faster than ever!

              Comment

              • George L
                Former vBulletin Support
                • May 2000
                • 32996
                • 3.8.x

                #8
                it outputs the 'No posts, please go back' message
                :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                :: Nginx SPDY SSL - World Flags Demo [video results]
                :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                Comment

                • George L
                  Former vBulletin Support
                  • May 2000
                  • 32996
                  • 3.8.x

                  #9
                  i ran this in phpmyadmin and it got mysql query error

                  Code:
                  SELECT title,threadid,lastpost FROM thread WHERE lastpost > '1136102400' AND lastpost < '1136620800' AND forumid='2000' ORDER BY dateline ASC
                  must be the forumid ? i only have 94 forums
                  Last edited by George L; Mon 30 Apr '01, 3:50pm.
                  :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                  :: Nginx SPDY SSL - World Flags Demo [video results]
                  :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                  Comment

                  • Overgrow
                    Senior Member
                    • May 2000
                    • 381

                    #10
                    It's just parsing the query string incorrectly, using the year instead of the forum. If you're having trouble like eva, change this:

                    Code:
                                $forumID=$urlArray[3];
                    	$year=$urlArray[4];
                    	$month=$urlArray[5];
                    	$week=$urlArray[6];
                    	$threadID=$urlArray[7];
                    to this

                    Code:
                                $a=0;
                    
                    	while($urlArray[$a] != "archive") {
                    		$a++;
                    	}
                    
                    	$forumID=$urlArray[($a+1)];
                    	$year=$urlArray[($a+2)];
                    	$month=$urlArray[($a+3)];
                    	$week=$urlArray[($a+4)];
                    	$threadID=$urlArray[($a+5)];
                    vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
                    Overgrow the Government: The Edge Forums
                    presented by Overgrow Marijuana Magazine


                    *warning: adult content since vB1.1... now faster than ever!

                    Comment

                    • George L
                      Former vBulletin Support
                      • May 2000
                      • 32996
                      • 3.8.x

                      #11
                      great works perfectly now

                      one last thing.. how do i prevent private forums from being listed/outputted and indexed ?

                      thanks much appreciated
                      Last edited by George L; Mon 30 Apr '01, 6:58pm.
                      :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                      :: Nginx SPDY SSL - World Flags Demo [video results]
                      :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                      Comment

                      • Chris Schreiber
                        Senior Member
                        • Jun 2000
                        • 4113
                        • 4.0.x

                        #12
                        Excellent and easy to install hack, thanks

                        I even liked the URL submission tool you linked to!

                        Comment

                        • Overgrow
                          Senior Member
                          • May 2000
                          • 381

                          #13
                          There ya go.. that is an important addition, eh?

                          There's a new zip file up top yonder.. the index.php has a new variable that holds your private forumid's.
                          vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
                          Overgrow the Government: The Edge Forums
                          presented by Overgrow Marijuana Magazine


                          *warning: adult content since vB1.1... now faster than ever!

                          Comment

                          • Brian
                            Senior Member
                            • Apr 2000
                            • 252

                            #14
                            One suggestion would be to show a suffix (.php , .shtml etc) so that is dynamic so the search engines know to spider it slower than if it was html. If they assume its html they might take on a tone at once as doing that to normal html files should be fine but this is actually doing all the db calls etc.

                            Just a suggestion but this is very nice!!

                            -Brian
                            << This Space Intentionally Left Blank >>

                            Comment

                            • Streicher
                              Senior Member
                              • Jan 2001
                              • 373
                              • 3.7.x

                              #15
                              I have tested your hack and my private forums are not hidden.

                              And also some thread are not found, by the index.php

                              When i click on some forumlinks the forumpage is simply reloaded.
                              Last edited by Streicher; Tue 1 May '01, 12:16am.
                              Streicher

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...