Sphinx search BETA for vBulletin 4.0

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • IB Adrian
    Former Senior Operations Manager
    • Jul 2008
    • 1688
    • 3.6.x

    Sphinx search BETA for vBulletin 4.0

    Sphinx search Beta for vBulletin 4.0
    Please read first:
    • Please do not install this product unless you have a strong technical background
    • We are not responsible for any conflicts or errors that occur with installing and/or operating sphinx
    • We recommend testing this product on a test instance of vBulletin first
    • This product is in Beta, and is not supported*
    • Do not upload your vbulletin-sphinx.php to a web accessible URL. Doing so would give away your database details. If you follow these exact instructions, it will not be in a web accessible URL.
    • The purpose of this product is to reduce database load (and reduce page load times) for sites that have large post databases and high activity. Boards that have less 4million posts and 3million pageviews a month will see a significantly reduced benefit from this product, and we see little need to install it on smaller sites that don’t exceed both of those metrics.
    • To install this sphinx search package, you must have the following things:
      1. Must have a Unix Server – package will not work with other server environments
      2. Have root access to the server
      3. Have a mySQL developer package (Yum, apt and RPM etc. – downloadable from mysql.com)
      4. GCC component (c++ compiler on host)

    • The product is to be installed on your webserver
    • Only install the version of sphinx specified in the instructions (newer and/or older versions may not function correctly)
    • Follow the instructions below exactly – you may need to make some adjustments based on “the flavor” of your unix system, and your installation of vBulletin.
    • This sphinx product attempts to replicate vBulletin default search functionality 100% to mirror user experience – so any bugs in the default vB4.0 search (or issues with its operation in general) will also occur with sphinx
    • If you would like to see an implementation of the product please go to www.bikeforums.net
    • If you adjust any of your config.php credentials you will need to restart your webserver/sphinx.
    • If you rebuild the threads in the admincp, you must re-run the main.sh
    • As of right now, there is no easy way of allowing this on more than one board per server. Should you want to, you would need to modify the sphinx config file, vbulletin config file, directories, cron files, etc. Also this is assuming you are running the Suite. It may work without but will throw an error on the indexing.
    • We have tried our best to offer as best guidance we can here in installing the product, and the product works on a number of instances of vbulletin with no known issues, but ultimately (as mentioned) we will not be responsible for any issues in installing it.
    • If you wish to edit (or remove) the "index throttle" to improve search index speed, edit the vbulletin-sphinx.php you can see it set here for:
      source ThreadPostMainSource : DBSource
      sql_range_step = 1024
      sql_ranged_throttle = 3000



    Install Sphinx
    --------------
    1. Login as root
    2. wget http://www.sphinxsearch.com/downloads/sphinx-0.9.8.tar.gz
    3. tar xzf sphinx-0.9.8.tar.gz
    4. cd sphinx-0.9.8
    5. ./configure --enable-id64 --prefix=/usr/local/sphinx
    6. make
    7. Did you get Error 127 during step 6? Install gcc-c++ using the following command and then repeat step 6.
      yum install gcc-c++
    8. make install
    9. Did you get dependency errors with steps 6 or 8? Install mysql-devel using the following command and then repeat steps 6 and 8.
      yum install mysql-devel



    Configure for vBulletin
    -----------------------
    1. Import product to vBulletin AdminCP’s Product Manager.
    2. Upload the contents of the upload folder to the vbulletin root directory.
    3. Assuming your forum is located in /home/vbulletin/public_html, Also you must define where your sphinx folder is located (different depending on where you ran the above commands). Then use this command:
      cp /path/to/sphinx-0.9.8/api/sphinxapi.php /home/vbulletin/public_html/packages/vbsphinxsearch
    4. At the end of your vbulletin config.php file, add the following:
      /*
      * Sphinx configuration parameters
      */
      $config['sphinx']['host'] = '127.0.0.1';
      $config['sphinx']['port'] = 3312;
      $config['sphinx']['path'] = '/usr/local/sphinx'; /* No trailing slash */
    5. Upload vbulletin-sphinx.php to /usr/local/sphinx/etc alongside the default sphinx.conf (May be named sphinx.conf.dist) file.
    6. Change $myforumroot in vbulletin-sphinx.php to the exact forum root (Example: /home/vbulletin/public_html).
    7. Set vbulletin-sphinx.php file to executable using:
      chmod +x /usr/local/sphinx/etc/vbulletin-sphinx.php
    8. Change directory to your sphinx folder using:
      cd /usr/local/sphinx
    9. Get the cron zip using:
      SEE TREVOR'S POST BELOW
    10. Unzip the cron.zip file using:
      SEE TREVOR'S POST BELOW
    11. Set all crons to executable using command:
      SEE TREVOR'S POST BELOW
    12. Run the main_norotate.sh cron using:
      /usr/local/sphinx/cron/main_norotate.sh
    13. Run the delta_norotate.sh cron using:
      /usr/local/sphinx/cron/delta_norotate.sh
    14. Start the daemon using:
      /usr/local/sphinx/bin/searchd --config /usr/local/sphinx/etc/vbulletin-sphinx.php
    15. To verify your Sphinx is working you can enter: ps ax | grep search[d]
      If the above returned something such as: /usr/local/sphinx/bin/searchd --config... the daemon is running.
    16. You should setup crons to run every 10 minutes on /usr/local/sphinx/cron/delta.sh and run /usr/local/sphinx/cron/main.sh once nightly.
    17. Go to your AdminCP->Options->Search Type. In the drop down, select Sphinx Search, then hit go.
    18. That is it! Sphinx should now be working correctly on your board.


    *This product is not supported via phone support, ticket support, or forum support. If there is sufficient interest in the product we may add a paid for support option for owners that are so interested. While we are continuing with development of this product (and will make new versions available as they are built), as it is a product that is utilized on a lot of Internet Brand’s own forums and are very vested in its future, we can provide no guarantee’s on future bug fixes and/or feature requests for the product, we will make every attempt in good faith to fix any bugs that arise and/or incorporate worthy suggestions that may benefit the product overall, but we are providing the product free of charge to assist a specific and small group of customers that run very large forums – and as a result the product is not considered a core vB competency, and its support is provided by developers that are outside the vB team (i.e. I have to call in favors ). While I hope that many of you will find this a worthy addition to vBulletin, I encourage you to please elect to download, install and utilize this product taking all of the above into account.


    We will be creating a forum on vbulletin.org for discussion on the future development of the sphinx search product. I will update this announcement with a link when that is complete.

    Thanks, and enjoy,
    Adrian
    Attached Files
    Last edited by Trevor Hannant; Wed 9 Jan '13, 9:49am.
    Adrian
  • IB Adrian
    Former Senior Operations Manager
    • Jul 2008
    • 1688
    • 3.6.x

    #2
    Forum for discussion of the product:
    This forum is for discussion and support of the free open source products from IB & vBulletin Solutions.
    Adrian

    Comment

    • Michael Biddle
      Senior Member
      • May 2004
      • 503
      • 3.8.x

      #3
      The zip file in the first post has been updated.

      The installation steps are the same as first post.

      Upgrade Instructions if you are already running the previous version:

      1. upload vbsphinxsearch folder into packages folder and set to overwrite

      2. Upload the new sphinx config file and re-set $myforumroot, also change the hashbang (Line 1 of sphinx config) if you had to already.

      3. Stop sphinx using:
      pkill searchd

      4. Ensure Sphinx has stopped running using:
      ps ax | grep search[d]

      5. Re-index main_norotate.sh and delta_norotate.sh using:
      /usr/local/sphinx/cron/main_norotate.sh
      /usr/local/sphinx/cron/delta_norotate.sh


      6. Start Sphinx using:
      /usr/local/sphinx/bin/searchd --config /usr/local/sphinx/etc/vbulletin-sphinx.php

      Bug Fix: Find post/threads by user
      Bug Fix: Improves performance of delta indexing

      Comment

      • Michael Biddle
        Senior Member
        • May 2004
        • 503
        • 3.8.x

        #4
        Bug Fix: Improved performance of delta indexing

        The zip in the first post and on carblog (Step 9 from the first post) have been updated.

        If you are already running our sphinx implementation, and wish to upgrade, please do the following:

        1. Overwrite all of the existing files within the /usr/local/sphinx/cron folder.
        2. Ovewrite the /usr/local/sphinx/etc/vbulletin-sphinx.php file from the vb4sphinx.zip file. Be sure that you edit the $myforumroot variable. Also depending on the server you are on you may need to edit the shebang on the first line.
        Last edited by Michael Biddle; Wed 17 Nov '10, 3:59pm.

        Comment

        • Trevor Hannant
          vBulletin Support
          • Aug 2002
          • 24325
          • 5.7.X

          #5
          The Cron ZIP is no longer available - instead, you will need to create your own Cron jobs:

          - Create a cron job to run once a week to reindex the entire index:

          /usr/local/sphinx/bin/indexer --config
          /usr/local/sphinx/etc/vbulletin-sphinx.conf test1main -rotate


          - Create a cron job to run every 10 minutes

          sudo /usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/vbulletin-sphinx.conf test1updates -rotate
          Vote for:

          - Admin Settable Paid Subscription Reminder Timeframe (vB6)
          - Add Admin ability to auto-subscribe users to specific channel(s) (vB6)

          Comment

          Related Topics

          Collapse

          Working...