What is SERVER configuration for vBulletin.com?

[rayzor]

I'm looking to set up the optimal SERVER for a large site with 2000-8000 members online at peak times.



I'd like to know,
What hardware configuration are you using, with 1000-1800 members onboard at most times, with hardly a hiccup (very fast)
with your most up-to-date vBulletin, what software (Centos/Apache/MySQL/PhP/cache, etc.) you're using and what special hardware considerations have you paid close attention to?Thanks in advance.

[eva2000]

is this a freshly started new vB forum or anticipated growth on an existing forum ? if it's the latter, what are the current usage/user online numbers and if you can provide info asked at http://www.vbulletin.com/forum/showthread.php?t=70117

If it's a new forum, then there's alot of factors to consider.

1. vB user online concurrently is based on default 15 min or 900 second cookie time out and is different from mysql concurrent connections ( max_connections/max_used_connections limit)

2. MySQL regardless of forum software, provided it has the right hardware and mysql, php and apache configuration is good for theoretically 1000 - 1500 and even 4,000 mysql concurrent connections (4,000 with the mysql staticly compiled rpm binaries)

vB users online figure is really deceptive in looking at forum usage loads. Consider this example of 3000 vB users over 900 second default cookie timeout versus over 7200 second cookie timeout would be completely different beasts in terms of usage.

Even 3000 vB users over 900 second cookie timeout can differ as you could have:


3000 vB users spread evenly i.e. 33.33 actual users every 10 seconds (probably lowest in terms of max mysql concurrent handling) or
3000 vB users where the first 120 seconds might have 800 vB users in total and then the remaining 780 seconds might have only ~28 users every 10 seconds
3000 vB users all within the same 5 second period actively using/search forum (result in the most mysql concurrent user handling requiring splitting servers)


All result 3000 vB users being reported but all would result in different ends of server loads and in terms of mysql load (max_used_connections/mysql conurrent users) and both would require different tweaks/hardware.

It may seem like overkill on some recommendations, but the point is you need hardware to handle the max/peak times/tasks not just the everyday loads.

If you had 3000 vB users over 9000 second cookie timeout (10x times longer than default), there would be an even greater variation as to the real server/mysql usage loads.

For example, 1 users active every 3 seconds over 9000 second duration is no way near as server intensive as 2500 users active over 1 second and remaining 500 users are active over remaining 8999 second duration.

3. It's not just hardware you need to consider.. some instances a single quad cpu server might not hold up as well as 2x dual cpu web + db servers as you also have software limitations i.e. linux/apache max open files and max file descriptor limitations, I/O disk cpu time/resources etc

there delays between connected servers is only minute but depends on how you configure them... using gigabit network cards via direct internal ip connection is most optimal compared to slower 100Mbp network cards etc

each persons configuration and forum/app usage patterns differs.. on same hardware one forum/community might not handle the same concurrency as another forum/community due to different traffic and usage patterns

Generally from working with 100s of large vB forums over past 7 yrs:
- around 250-350 mysql concurrent connections your looking at dual cpu server at least mid range specs dual p4 2.4-2.8ghz xeons or dual opteron 240-242 (the better of the 2)
- around 300-450 mysql concurrent connections you're looking to plan to split to 2x dual cpu server setup where you have a dedicated web server connected to a dedicated mysql database server
- at around 600-900+ mysql concurrent connections minimum spec would be 2x dual cpu server setup with at least dual p4 3.4-3.6ghz xeons and 4GB of ram on mysql server or better performance to price ratio http://www.vbulletin.com/forum/showthread.php?t=160103 dual opteron 250 cpus with scsi disks in raid 10. The dual intel xeon 51xx woodcrest /xeon 53xx clovertown servers are currently the fastest servers to get.

Remember this is mysql concurrent connections not vB users online as explained above.

Most vB forums I've looked at even the large ones at 2,000 to 8,000 vB users online reported really only have mysql concurrent connection usage between 450-900 mysql concurrent connections.. I've seen a few with other mysql apps contributing in hitting 1,200 to 1,500 mysql concurrent connections... As I said forum vB user patterns can vary when translated in more meaningful mysql concurrent connections usage.

Most of the above is with apache web server... lighttpd and litespeed web server tends to handle mysql concurrent connection usage better than apache so can skew the results somewhat.



Now with all that being said, for your needs, I'd generally start off with the following:

A multi dual/quad cpu servers setup consisting of load balanced web servers connected internally via gigabit network cards to a dedicated mysql database server. I would have vB attachments disabled so as to allow load balanced web servers. vB attachments are stored in database by default but for optimal performance should be moved into filesystem via vB admin control panel. But being on filesystem a load balanced web server would need to be hacked to sync file attachments across several load balanced web servers. Having vB attachments disabled would be the easy way to do this for load balancing unless you want to hack the vB forum software code.

For web server end I would use a hardware load balancer and start with one or two web servers and add more servers later as needed.

Each web server consisting of the following:

- Dual Xeon 5420/5430 Harpertown Quad core (2 quad = 8 cores)
- 4x1GB minimum memory or 4x2GB - more memory will allow you to handle more concurrent apache connections
- 10K SCSI/SAS raid 10 for speed and redundancy optimal setup if data and/or minimal downtime is important
- dual gigabit network cards as outlined at as outlined at http://www.vbulletin.org/forum/showthread.php?t=111191
- RHEL 64bit OS
- PHP 4.4.8 or 5.2.5
- Xcache v1.2.2 http://xcache.lighttpd.net/
- Apache 2.2.x or if you prefer Lightspeed Enterprise web server http://litespeedtech.com/products/webserver/overview/ which is around 9x faster than Apache but cost $$$. Or lighttpd or nginx web serving software.

A dedicated mysql database server consisting of the following:
- Dual Xeon 5420/5430 Harpertown Quad core (2 quad = 8 cores)
- 8GB minimum
- 15K SCSI/SAS raid 10 for speed and redundancy optimal setup if data and/or minimal downtime is important
- Dedicated hardware SCSI/SAS raid controller with onboard processore and at least 256MB cache and battery backup
- dual gigabit network cards as outlined at as outlined at http://www.vbulletin.org/forum/showthread.php?t=111191 - second network card in case you want to set up a second mysql database server to use as a MySQL replication slave server
- RHEL 64bit OS
- MySQL 5.0.51 64bit statically compiled if problems try 64bit rpm binaries

Then you'll need settings for apache/mysql/php tweaked for optimal use - questions or info needed for me to optimise them are outlined at http://www.vbulletin.com/forum/showthread.php?t=70117

This outlined setup would be what I'd go with to handle 2000+ vB users, but there isn't any guarantee as to how much it will handle as outlined above, vB users online is not the best way to gauge load. You could handling 5,000 vB users online depending on traffic patterns and cookie timeout set. The real factor is mysql concurrent user activity.

Hope this helps

[rayzor]

Thanks Eva for a very thorough explanation.

Looking at the historical Apache Web SERVER clients graphs in InterWorx panel, I see a consistent 200 to 220 processes almost on a daily basis.

For MySQL, I see an average of 140 simultaneous queries.

To do the actual software install with Lightspeed and all the other components on a dedicated SERVER, should I go to vbulletin.org and hire an expert who can do this all for me, as well as installing and configuring the latest vbulletin and importing (perhaps via impex) the current 3.5.0 DB, or do you have any other recommendations?

[eva2000]

lightspeed has support contracts and support forum to help with installs http://www.litespeedtech.com/support/services/ maybe ask there first :)