Bulletin Board Benchmarks: http://bb-bench.com

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • ckwalsh
    New Member
    • Sep 2010
    • 4

    Bulletin Board Benchmarks: http://bb-bench.com

    I was tired of anecdotal performance comparisons for various forum software, so I decided to put together a site to coordinate some benchmarks, located at http://bb-bench.com. The most recent benchmark is at http://bb-bench.com/benchmark/6, and includes vBulletin 3.8 and 4.0, as well as several other leading bulleting board softwares.

    I'd appreciate any comments on my findings. I started with a phpBB3 forum and converted to all other forum software, to duplicate the content. Each piece of software was installed under the default settings, and optimized using obvious configuration options (aka, I haven't researched the optimal settings for every installation), and have also taken suggestions from vendors for tweaks. They were run on the same database backend, and each had their sessions tables cleared before each run.

    Looking at the results, you will also see data for phpBB with HipHop for php. I also have done some work in making phpBB3 compatible with HipHop for PHP, but it is specially labeled because the comparison is not fair; HipHop uses its own server and is compiled. However, it does show that phpBB3 can be used with HipHop, and the advantage HipHop offers.

    Just for full disclosure: I am on the Moderator Team on phpBB.com

    I'm glad vBulletin allows such discussion, as mentioned in your rules ("While we are quite happy to have constructive comparative conversations about competitor's products, we will not tolerate blatant adverts or the sort of 'ooh look, yet another bulletin board' threads. Also, we would appreciate it if you refrain from blatant bashing of other boards."), and I hope you can suggest tweaks and optimizations to improve these benchmarks.

    Thank you

    ckwalsh
  • Zachery
    Former vBulletin Support
    • Jul 2002
    • 59097

    #2
    I'd be curious to know if you're using any datastore cacheing in vBulletin? Or the other softwares? moving data into shared memory can greatly enhance how fast pages can be delivered.

    Is there any reason you're not using nginx or lighttpd as well as PHP-FPM?
    Is mysql being optimized at all? How is it configured?
    Also slightly curious of the OS and its configuration.

    Comment

    • Lizard King
      Banned
      • Mar 2004
      • 1891
      • 3.6.x

      #3
      I am curious to see why vBulletin requires special server configuration for better performance. vBulletin should perform perfect with an unoptimized server at all. Optimization should just improve the performance you receive.

      Comment

      • ckwalsh
        New Member
        • Sep 2010
        • 4

        #4
        Originally posted by Zachery
        I'd be curious to know if you're using any datastore cacheing in vBulletin? Or the other softwares? moving data into shared memory can greatly enhance how fast pages can be delivered.

        Is there any reason you're not using nginx or lighttpd as well as PHP-FPM?
        Is mysql being optimized at all? How is it configured?
        Also slightly curious of the OS and its configuration.
        I found the APC datastore caching option for vBulletin, and enabled it when I performed the install. Since the APC opcode cache is being used, the all the bytecode is being stored in memory rather than being read from disk.

        As for not using nginx/php-fpm, I started out using them and found they were unstable on this machine. Now, I'm 99.9% this is a configuration error I made at some point, or an installation mistake, but I wasn't able to find anything obvious and decided to stick with Apache. Last time I tested apache vs nginx (with the bundled fastcgi server, not php-fpm admittedly), the differences were barely noticeable, compared to actual page generation time.

        The machine is running Ubuntu 10.04, Server edition on a local intranet, so doesn't have any additional load. As for MySQL, I won't claim to be a db wizard, but I tried my best to make sure memory was not an issue (heck, I have 12gb to throw around). Here's my config, with comments removed for brevity - I wouldn't mind if you could make suggestions.

        Code:
        [client]
        port            = 3306
        socket          = /var/run/mysqld/mysqld.sock
        
        [mysqld_safe]
        socket          = /var/run/mysqld/mysqld.sock
        nice            = 0
        
        [mysqld]
        user            = mysql
        socket          = /var/run/mysqld/mysqld.sock
        port            = 3306
        basedir         = /usr
        datadir         = /var/lib/mysql
        tmpdir          = /tmp
        skip-external-locking
        
        key_buffer              = 1024M
        max_allowed_packet      = 16M
        thread_stack            = 192K
        thread_cache_size       = 8
        myisam-recover         = BACKUP
        table_cache            = 64
        
        query_cache_limit       = 1M
        query_cache_size        = 128M
        log_error                = /var/log/mysql/error.log
        
        
        expire_logs_days        = 10
        max_binlog_size         = 100M
        
        [mysqldump]
        quick
        quote-names
        max_allowed_packet      = 16M
        
        [mysql]
        
        [isamchk]
        key_buffer              = 512M

        Comment

        • ckwalsh
          New Member
          • Sep 2010
          • 4

          #5
          All software has tweaks that can be made to improve performance - I'd be rather surprised if the community cannot suggest a few improvements.

          Originally posted by Lizard King
          Optimization should just improve the performance you receive.
          That's my point. Surely board admins will make as many optimizations as reasonable for their server configuration - I am trying to mimic that. If you don't want to do anything and let other vendors suggest changes to improve their response times, I guess that's all right, but incredibly silly.

          Comment

          • Zachery
            Former vBulletin Support
            • Jul 2002
            • 59097

            #6
            Originally posted by Lizard King
            I am curious to see why vBulletin requires special server configuration for better performance. vBulletin should perform perfect with an unoptimized server at all. Optimization should just improve the performance you receive.
            More curious why he wouldn't run a more optimized setup as a whole I'm not aware of other softwares cacheing methods or special configurations

            Comment

            • ckwalsh
              New Member
              • Sep 2010
              • 4

              #7
              Originally posted by Zachery
              More curious why he wouldn't run a more optimized setup as a whole I'm not aware of other softwares cacheing methods or special configurations
              I attempted to optimize the entire system to the best of my ability, and made additional changes to each individual software.

              [SARCASM]I guess I could apply those fairly too. How do I enable the phpBB3 APC cache plugin for vBulletin? It sounds like a great idea, I just can't find config.php. Nor can I figure out where to disable template eval in vBulletin, which the smf developers recommended; that key isn't in the vB config table. Any ideas?[/SARCASM]

              Surely there are tweaks you recommend to some of your large clients to improve their performance. I was hoping to get pointed to some of them, or at least generate some discussion on the results. If you are only jumping on my phrasing for "optimizations", good day to you.

              Comment

              • Zachery
                Former vBulletin Support
                • Jul 2002
                • 59097

                #8
                I'm not aware of any way to get rid of eval for vBulletin's templates.
                The APC shared memory is a setting in the config file though

                Edit, wasn't jumping on you for not optimizing. Just curious as to what you had done and if you had tried anything. I've worked with bigger clients and squeezing preformance out of everything can make a lot of things count for how preformance handles.

                I'd actually be more intrested how the load continues to scale at good intervals of users.
                Last edited by Zachery; Mon 20 Sep '10, 11:13am.

                Comment

                • IBxAnders
                  Senior Member
                  • Aug 2001
                  • 1172
                  • 4.0.x

                  #9
                  ckwalsh; what are you using to test exactly, mysqlslap, siege, how are you loading the databases? (sorry if you've already gone over this)..

                  Are you also invalidating cache? I have some suggestions and interest in adding some more tests, at least for vBulletin, in case you are interested in maybe working together on some tests.
                  anders | vbulletin team | check out the new vbulletin facebook app
                  Proudly vBulletin'ing since 2001
                  Please be my friend!
                  http://www.twitter.com/inetskunkworks
                  vBulletin Performance Articles:
                  Click here to read

                  Comment

                  • Abomination
                    Senior Member
                    • Jul 2008
                    • 1244

                    #10
                    The numbers look right to my experiences. Especially the higher standard deviation for the board index for ipb.

                    But I have not run any benchmarks.

                    Comment

                    widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                    Working...