The success of the archive hack

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • SkuZZy
    Senior Member
    • Aug 2002
    • 447

    The success of the archive hack

    http://www.vbulletin.org/forum/showt...threadid=47087


    Hey all,

    Some of you might remember a few months back that I created a hack (in conjunction with Xenon, who rocks!) that's sole purpous was to get posts and threads spidered. That was in December. With the Febuary "google dance" just finishing up, just wanted to share the results of my hack with all of you. it shows that in 3 months i've gotten over 17,000 pages spidered into google, and it's increased my hits alot.




    Anyways, i'm mainly just posting here to let anyone know who is looking for a way to get their posts spidered, that the hack's Final version has been released. You can download it over at vb.org at:





    You can view some demos of customized archives here:

    http://www.destee.com/forums/archives
    http://www.thrillnetwork.com/boards/archive/
    http://www.truecountry.com/forums/history/


    The final version supports a bunch of stuff, including multiple pages now (threads with over 200 posts are split up into multiple pages) and sub-forums also. As this is the final version, no other releases will be made, especially since the vb3 forums is coming soon (and great job to the staff for that). Those of you, however, who plan on sticking with vb2, can still have an archive, using these scripts. Good things about these scripts are that they require no modifcations to any of the scripts OR templates, just upload the files, modify 3 variables in the config and you're set.

    SkuZZy
    Last edited by SkuZZy; Tue 18 Mar '03, 9:33pm.
    -
    Visit the Web Scripts Directory @ http://www.scriptz.com
    -
    PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

    -
  • IDN
    Senior Member
    • Apr 2002
    • 4030
    • 3.5.x

    #2
    I added it to my sex forum, so i should get major hits from perverts, which has the good and the bad...

    Submitted my forum to the engines the day i opened my site, google started to spider on saturday, and i'm still not listed in google
    Running vB since 4-14-2002

    Comment

    • ManagerJosh
      Senior Member
      • Jun 2002
      • 9922

      #3
      Takes about one week for google to add your site.
      ManagerJosh, Owner of 4 XenForo Licenses, 1 vBulletin Legacy License, 1 Internet Brands Suite License
      Director, WorldSims.org | Gaming Hosting Administrator, SimGames.net, Urban Online Entertainment

      Comment

      • IDN
        Senior Member
        • Apr 2002
        • 4030
        • 3.5.x

        #4
        For some reason, the archive hack is the same as the old version, you didn't updated it really, did you?
        Running vB since 4-14-2002

        Comment

        • bromyaur
          Senior Member
          • Jan 2002
          • 232
          • 3.6.x

          #5
          I installed the archive about a month ago (middle of february), I always have about 30 to 90 bots on my forum everyday, still waiting to see the results on google....hopefully soon.

          Comment

          • SkuZZy
            Senior Member
            • Aug 2002
            • 447

            #6
            Originally posted by IDN
            For some reason, the archive hack is the same as the old version, you didn't updated it really, did you?
            The final version was released, I believe, about 2 weeks ago. I just never got around to letting everyone here know.
            -
            Visit the Web Scripts Directory @ http://www.scriptz.com
            -
            PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

            -

            Comment

            • IDN
              Senior Member
              • Apr 2002
              • 4030
              • 3.5.x

              #7
              Made me run there for nothing, bad boy!
              Running vB since 4-14-2002

              Comment

              • Overgrow
                Senior Member
                • May 2000
                • 381

                #8
                It can take over a month to get into Google. If you have an incoming link from a site that's marked "Fresh" (a date next to their result), then they are spidered every 2 or 3 days, and there is a chance that your site will get slurped up by the FreshBot as well. It's much more normal for the delay to be 3 to 6 weeks as the DeepCrawlers find you and add you to the real database.

                I've got over 50,000 Google page listings now.. Since this thread is full of horns and blowing, here is a little article I wrote about converting all of those SERP visitors into forum members:



                Check out my archive for an example of what I think is a good listing that should draw the searchers into the rest of the site. If your archive doesn't do that, what good is it right? Also, it's got lots of good internal links to boost my PR.



                Thanks for the hack skuzzy! I wrote vBSpiderFriend, which I don't use anymore, and I'm not using yours, but they all do basically the same thing.. it's the end-user's design that makes the difference.
                vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
                Overgrow the Government: The Edge Forums
                presented by Overgrow Marijuana Magazine


                *warning: adult content since vB1.1... now faster than ever!

                Comment

                • SkuZZy
                  Senior Member
                  • Aug 2002
                  • 447

                  #9
                  Originally posted by IDN
                  Made me run there for nothing, bad boy!
                  sowwie
                  -
                  Visit the Web Scripts Directory @ http://www.scriptz.com
                  -
                  PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

                  -

                  Comment

                  • 0ptima
                    Senior Member
                    • Jan 2002
                    • 1557

                    #10
                    Before VB, I used WWWBoard and Anyboard which created static html files which I have kept as archives. I have about 2 years worth of archives and Google has indexed 16,900 pages! Certain search phrases list my site near the top in Google and they are not directly related to my site (One particular phrase lists my site above the manufacturers website!)

                    Once VB3 comes out, I will install this hack if it is available.

                    Comment

                    • dpkm
                      Member
                      • Dec 2002
                      • 55
                      • 3.6.x

                      #11
                      Optima, no need to install this hack with vB3 - vB3 comes with an archive (see bottom of the page).

                      I checked out the script, and it looks great - but I thought I should remind everyone that as easy as this script makes it to get your forums spidered by Google, you should also consider how difficult it is to get the old URLs _out_ of Google if/when you switch to vB3 and move to the vB3 archiving system.

                      Potentially, you could have thousands of search engine results pointing to your site, all pointing to 404s - and 404s can stick around a few Google update cycles. A 301 permanent redirect (as Google recommends to get old pages out of the index in the first go-round) for the whole archive directory will force your site visitors to go through extra steps to find what they were looking for, and may put people off. It would all be sorted out by Google eventually, but at what cost?

                      I don't want to diminish what's been done - I just thought people should consider the above.
                      Last edited by dpkm; Tue 25 Mar '03, 8:38pm.

                      Comment

                      • Overgrow
                        Senior Member
                        • May 2000
                        • 381

                        #12
                        Good points, dpkm. On the first digest, Google picked up my chat forums and now I hit top 5 for phrases like "christina agulera naked" and "gta vice city hidden packages" ... I have these forums blocked from the archive now and show those incoming visitors a "no permission" sign (with banners of course).
                        vB Retro -> See vBulletin 1.1.2 and travel back to the Year 2000!
                        Overgrow the Government: The Edge Forums
                        presented by Overgrow Marijuana Magazine


                        *warning: adult content since vB1.1... now faster than ever!

                        Comment

                        • Dave#
                          Senior Member
                          • Jul 2000
                          • 1845

                          #13
                          Why would anyone get 404's?

                          The syntax of the url remains the same:



                          So the pages generated by either the hack or VB archive will point to the correct content?

                          Or am I missing something?
                          http://forums.cpfc.org/

                          Comment

                          • dpkm
                            Member
                            • Dec 2002
                            • 55
                            • 3.6.x

                            #14
                            Originally posted by Dave#
                            Why would anyone get 404's?

                            The syntax of the url remains the same:



                            So the pages generated by either the hack or VB archive will point to the correct content?

                            Or am I missing something?
                            Hmm.. well, unless I am missing something (and it's possible!) both the hack and vB3 archive generate search engine friendly pages and links that only exist when a request is made (that is, they do not exist as separate files on the web server), and those SE-friendly pages and links are different than the actual URL.

                            e.g., the true URL : http://www.vbulletin.com/forum/showt...threadid=67798

                            and the generated URL:


                            It is the generated URL that gets indexed by the search engine, and if you create a suite of generated URLs with the hack, you will need to ensure that your generated URLs are exactly the same when you implement vB3 (and it is certainly possible, but I think it will require some work). If you use the hack and then switch to vB3 and its archive with different URLs, then the hack URLs will return 404s to your visitors from search engines (until the search engines reindex everything and drop the old 404s which can take some time in some cases).
                            Last edited by dpkm; Wed 26 Mar '03, 9:30pm.

                            Comment

                            • SkuZZy
                              Senior Member
                              • Aug 2002
                              • 447

                              #15
                              Just wanted to say, with the newest update, battleforums now has 20k+ pages in google: http://www3.google.com/search?hl=en&...m+battleforums

                              As for the vb3 archive, it doesn't work for betting your posts spidered in google. And someone mentioned above that when you switch to vb3, the old links will all be 404's. That can be fixed with a .htaccess that will redirect users to the proper thread. It can be easily done, or could even just make it so your 404 error page goes directly to your home page.

                              Edit - Actually, with the latest update, it looks like the vb3 archive does work, somewhat. It isn't exactly "optimized" though... http://www.google.com/search?q=site:...start=120&sa=N - Don't know if google likes it or not.
                              -
                              Visit the Web Scripts Directory @ http://www.scriptz.com
                              -
                              PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

                              -

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...