vB4 Search Engine

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Kevlar
    Senior Member
    • Aug 2001
    • 393
    • 3.8.x

    [Forum] vB4 Search Engine

    I remember reading somewhere that for vB4, the engine search engine process and storage features were rewritten to work better with large forums. Unfortunately, I do not remember where I read this. I am curious to see if the new vB4 search is capable of working with larger forums (15 million+ posts) or if I have to look for another solution like I am currently using with 3.8.4?
    Kevlar
    Bimmerforums - The Ultimate BMW Forum
  • Dody
    Senior Member
    • Aug 2004
    • 1896
    • 3.8.x

    #2
    vB 4.0 is just the first step, I advice you to stick around and wait. Newer versions and updates will bring a better searching functionality and others.
    while(true){
    if(
    $someone->needsHelp() && $i->canHelp()) $post->help();
    if(
    $i->findBug()) $post->bug();
    }

    Comment

    • Kevlar
      Senior Member
      • Aug 2001
      • 393
      • 3.8.x

      #3
      True... but that doesn't help me now if I want to upgrade from 3.8.4 to 4.0. If 4.0 can't do the searching, I'll have to hack together another way of doing it like I am doing now.
      Kevlar
      Bimmerforums - The Ultimate BMW Forum

      Comment

      • Wayne Luke
        vBulletin Technical Support Lead
        • Aug 2000
        • 73981

        #4
        Search was rewritten. The primary focus was getting search out of the content tables and into its own indexed tables. This allows you to switch your post and thread tables to a system like INNODB and remove the table level locks on them. Removing the locks also allows more writes to happen. Another change is that the search interface is completely modular so a new interface can be plugged into the system easier if need be. As Internet Brands converts their larger forums, I am sure additional refinements and features will be added.

        This blog goes over some of the issues: http://www.vbulletin.com/forum/entry...nges-in-vB-4-0
        Translations provided by Google.

        Wayne Luke
        The Rabid Badger - a vBulletin Cloud demonstration site.
        vBulletin 5 API

        Comment

        • kontrabass
          Senior Member
          • Jul 2000
          • 302
          • 3.6.x

          #5
          Wayne,

          I know you're hesitant to give out details, but there are a lot of big-boarders out there who are anxious to know what to do. 4.0 looks great, but counting on mysql to search just doesn't cut it. For example I just searched vb.com for 'sphinx' and the search took over 31 seconds. With 2,000 members online at once, I can't imagine how such searches would be useable on our board.

          So, I guess the question is... should we get to work porting Sphinx over to 4.0, or should we wait for a future point release of VB? If VB has no intention of incorporating Sphinx or other workable search, we'd better get to work on our end. We've been with VBulletin for 10 years and are dying to upgrade to 4.0 - but since 2007 have completely relied on Sphinx to make VBulletin a usable product.

          Comment

          • Kevlar
            Senior Member
            • Aug 2001
            • 393
            • 3.8.x

            #6
            Originally posted by Wayne Luke
            Search was rewritten. The primary focus was getting search out of the content tables and into its own indexed tables. This allows you to switch your post and thread tables to a system like INNODB and remove the table level locks on them. Removing the locks also allows more writes to happen. Another change is that the search interface is completely modular so a new interface can be plugged into the system easier if need be. As Internet Brands converts their larger forums, I am sure additional refinements and features will be added.

            This blog goes over some of the issues: http://www.vbulletin.com/forum/entry...nges-in-vB-4-0
            Thank you for the information... but that really doesn't give me anything concrete to base moving a large forum over to the new platform. My license to vB4 has already been purchased... but I am in a holding pattern until I can figure out how I am going to get search to work effectively.

            Originally posted by kontrabass
            Wayne,

            I know you're hesitant to give out details, but there are a lot of big-boarders out there who are anxious to know what to do. 4.0 looks great, but counting on mysql to search just doesn't cut it. For example I just searched vb.com for 'sphinx' and the search took over 31 seconds. With 2,000 members online at once, I can't imagine how such searches would be useable on our board.

            So, I guess the question is... should we get to work porting Sphinx over to 4.0, or should we wait for a future point release of VB? If VB has no intention of incorporating Sphinx or other workable search, we'd better get to work on our end. We've been with VBulletin for 10 years and are dying to upgrade to 4.0 - but since 2007 have completely relied on Sphinx to make VBulletin a usable product.
            Basically yes... right now I'm using Sphinx for advanced search and I would like to know if I have to figure out how to make it work for vB4 or if I can rest assured that the new changes to the vB4 search engine will be all I need. The answer to that question will depend on if I upgrade tomorrow or two months from now.
            Kevlar
            Bimmerforums - The Ultimate BMW Forum

            Comment

            • Kevlar
              Senior Member
              • Aug 2001
              • 393
              • 3.8.x

              #7
              Does 4.0.1 bring any new insights to what search can do for big forums?
              Kevlar
              Bimmerforums - The Ultimate BMW Forum

              Comment

              • Andy
                Senior Member
                • Jan 2002
                • 5886
                • 4.1.x

                #8
                Originally posted by Wayne Luke
                Another change is that the search interface is completely modular so a new interface can be plugged into the system easier if need be.
                Hi Wayne,

                I found this very interesting. I have used my own search engine for years. So I would really like to see an easier way to use another search engine without having to make dozens of template edits. What would be nice is to be able to have a few entries in the Admin CP that would allow us to substitute custom url's for search items like theses:

                1) Find all Posts by user
                2) Find all Threads by user
                3) Search Forum
                4) Search Thread

                Thank you.

                Comment

                • floid78
                  New Member
                  • Jan 2008
                  • 11
                  • 3.6.x

                  #9
                  I'm as well dependent on either Sphinx for vb4 or a decent solution for search on big boards for vb4 in a future release.

                  Regarding Sphinx there was a promising statement made by Kevin Sour in a recent developer chat which seems to be overlooked so far:

                  We have a sphinx search working in house here at IB. Its not yet ready to release to the wild, but we do have plans to clean it up and get it out there in some form.



                  It would be very nice to get any update on this.

                  Comment

                  • legolas
                    Member
                    • Jan 2003
                    • 84
                    • 3.8.x

                    #10
                    According to this dev chat, you had a sphinx search plugin working in November, and now three months later it's still not cleaned up and released?

                    Do you not realize how critical fast searching is for a lot of us? It turns a forum that requires split db servers (to isolate slow search queries) into a forum that works well on a single server with modest hardware, because the search is blazing fast and doesn't slow down everything else.

                    The lack of sphinx search integration is the primary reason I haven't upgraded to vb4. I don't want to put effort into updating the existing sphinx hack only to have you release an officially supported vb4 sphinx solution soon after.

                    Please release the sphinx search module already.

                    Thanks.

                    Comment

                    • rossco_2005
                      Senior Member
                      • Apr 2005
                      • 465
                      • 4.1.x

                      #11
                      Originally posted by legolas
                      According to this dev chat, you had a sphinx search plugin working in November, and now three months later it's still not cleaned up and released?

                      Do you not realize how critical fast searching is for a lot of us? It turns a forum that requires split db servers (to isolate slow search queries) into a forum that works well on a single server with modest hardware, because the search is blazing fast and doesn't slow down everything else.

                      The lack of sphinx search integration is the primary reason I haven't upgraded to vb4. I don't want to put effort into updating the existing sphinx hack only to have you release an officially supported vb4 sphinx solution soon after.

                      Please release the sphinx search module already.

                      Thanks.
                      I'm in the same boat.

                      http://filesharingtalk.com

                      Comment

                      • kontrabass
                        Senior Member
                        • Jul 2000
                        • 302
                        • 3.6.x

                        #12
                        I'm sorry, but after not even a 'boo' from any vb staff in reply to any of our queries I'm about to jump ship. Which is sad after 10 years with vb

                        Comment

                        • Kevlar
                          Senior Member
                          • Aug 2001
                          • 393
                          • 3.8.x

                          #13
                          Originally posted by rossco_2005
                          I'm in the same boat.
                          I'm in the boat too... I purchased vB4 with the assumption that it would work for my needs. Sadly, it looks like I have been mistaken. Month old promises without any word on a real solution are doing nothing but pushing me to find another piece of software, especially since right now I have a license for a piece of software I can't use for anything more than a paperweight.
                          Kevlar
                          Bimmerforums - The Ultimate BMW Forum

                          Comment

                          • ivanp
                            Senior Member
                            • Sep 2007
                            • 148
                            • 4.2.x

                            #14
                            Exactly, without Sphinx we cannot upgrade to vb4, also.

                            They said in November it is in beta stage and still nothing. Not even a status update.

                            Did we pay vb4 license for nothing?

                            Comment

                            • digitalpoint
                              Senior Member
                              • Mar 2004
                              • 2573
                              • 4.1.x

                              #15
                              For those that asked, the default vB4 search isn't acceptable for a "big board". We upgraded on Saturday and the process that goes through and reindexes old posts is still running (it's been running 5 days now).

                              Some of the simplest searches take a few minutes to run. Then on top of that, when those queries are running, it locks up the replication thread on the slave server doing the query (see this post/example). A search that causes the entier site to only be as up to date as when the search started (for all users) isn't really that good.

                              I wish someone would get Sphinx working with vB4 so I don't have to do it myself.
                              Sphinx Search for vBulletin 4: https://marketplace.digitalpoint.com...tin-4.870/item
                              Someone send me a message on Twitter when this site is usable again. https://twitter.com/digitalpoint

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...