Optimal Server design for Vb3.x...RAM? CPU's?

[tribedude]

We are going to be getting another server soon.

We currently have over 2 million posts, 500 concurrent users on our existing machine and it is running well, touch wood (tapping head).

However, I hope to get something with room for growth, perhaps even starting another, different themed community on it in addition to our main one.

I am curently looking at a Dell 2850 with 2x 3.0 Xeons...

Are faster chips preferrable to more RAM? Or is it the other way around... What is the best balance of RAM and chip speed for vB?

Also, our current machine doesn't use Raid. Should we use raid on the new machine? How does Raid affect the speed of vB?

Finally, I understand that splitting a board across 2 machines is doable... Using one to serve the pages and another for the database... What are the rules about this? How should a 2 server setup like this be designed for maximum speed and solidity..?

Thanks,

D

[tribedude]

Oh, I forgot... What about opteron servers.... Are they as good as Xeons for vB?

[eva2000]

what's your current server config and forum info as asked at http://www.vbulletin.com/forum/showthread.php?t=70117 ?

dual AMD64 opterons are beasts that run alot faster for mysql server/database tasks so if you have 2nd server use dual AMD64 opterons for mysql database server i.e. dual AMD64 opteron 244-246s would be nice with 4GB ram on database server and scsi 15k rpm raid 10 or raid 5

[tribedude]

what's your current server config and forum info as asked at http://www.vbulletin.com/forum/showthread.php?t=70117 ?

dual AMD64 opterons are beasts that run alot faster for mysql server/database tasks so if you have 2nd server use dual AMD64 opterons for mysql database server i.e. dual AMD64 opteron 244-246s would be nice with 4GB ram on database server and scsi 15k rpm raid 10 or raid 5


George how should a dual server setup generally work? Should the faster machine serve the webpages or be the MYSQL/dtabase server? We are currently using a DEll dual tuey 1.26 with 2 gigs of ram and it is nice as I say but it is getting old. Would that machine be ufficient to serve the webpages if we build an opteron server?

[BamaStangGuy]

George how should a dual server setup generally work? Should the faster machine serve the webpages or be the MYSQL/dtabase server? We are currently using a DEll dual tuey 1.26 with 2 gigs of ram and it is nice as I say but it is getting old. Would that machine be ufficient to serve the webpages if we build an opteron server?
He needs the info in the link

[tribedude]

He needs the info in the link

I am not sure what you mean...

I am not looking for optimizing info on our existing box (already done, thanks vB board), I am looking for tips on what boxes to buy for an ideal single server or dual server setup that just for vB

[BamaStangGuy]

I am not sure what you mean...

I am not looking for optimizing info on our existing box (already done, thanks vB board), I am looking for tips on what boxes to buy for an ideal single server or dual server setup that just for vB
He needs that info so he can look at your stats and suggest a server setup for you

[tribedude]

here it is George:

1. Is this on dedicated or shared virual server.
Dedicated server

2. your server specs. For example:

cpu speed/type single or dual cpus): dual tualin 1.26
how much memory installed: 2 gig
hard drive type/configuration: 2 x 36 gig 15k SCSI (Non-Raid), One of the drives is for a backup dump
linux distributor: fedora 2.6.5-1.358smp
apache/IIS version: apache ?
PHP version: php 4.3.10
MySQL version: mysql 3.23.58


3. if you use mysql 4.x instead of mysql 3.23.x, do you have any innodb type databases/tables on your server ????

4. if possible how mysql was compiled/installed????

5. your top stats
top - 00:33:27 up 200 days, 6:23, 1 user, load average: 0.76, 0.69, 0.71
Tasks: 144 total, 7 running, 137 sleeping, 0 stopped, 0 zombie
Cpu(s): 46.1% us, 10.9% sy, 0.0% ni, 41.8% id, 1.0% wa, 0.0% hi, 0.2% si
Mem: 2076008k total, 2042804k used, 33204k free, 48916k buffers
Swap: 2040244k total, 0k used, 2040244k free, 1551644k cached

6. your mysql configuration variables

| back_log | 50 |
| basedir | / |
| binlog_cache_size | 32768 |
| bulk_insert_buffer_size | 8388608 |
| character_set | latin1 |
| character_sets | latin1 big5 czech euc_kr gb2312 gbk latin1_de sjis tis620 ujis dec8 dos german1 hp8 koi8_ru latin2 swe7 usa7 cp1251 danish hebrew win1251 estonia hungarian koi8_ukr win1251ukr greek win1250 croat cp1257 latin5 |
| concurrent_insert | ON |
| connect_timeout | 10 |
| convert_character_set | |
| datadir | /var/lib/mysql/ |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_min_word_len | 4 |
| ft_max_word_len | 254 |
| ft_max_word_len_for_sort | 20 |
| ft_stopword_file | (built-in) |
| have_bdb | NO |
| have_crypt | YES |
| have_innodb | YES |
| have_isam | YES |
| have_raid | NO |
| have_symlink | YES |
| have_openssl | NO |
| have_query_cache | YES |
| init_file | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_buffer_pool_size | 8388608 |
| innodb_data_file_path | ibdata1:10M:autoextend |
| innodb_data_home_dir | |
| innodb_file_io_threads | 4 |
| innodb_force_recovery | 0 |
| innodb_thread_concurrency | 8 |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_fast_shutdown | ON |
| innodb_flush_method | |
| innodb_lock_wait_timeout | 50 |
| innodb_log_arch_dir | ./ |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 1048576 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | ./ |
| innodb_mirrored_log_groups | 1 |
| innodb_max_dirty_pages_pct | 90 |
| interactive_timeout | 28800 |
| join_buffer_size | 1044480 |
| key_buffer_size | 16777216 |
| language | /usr/share/mysql/english/ |
| large_files_support | ON |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_update | OFF |
| log_bin | OFF |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_warnings | ON |
| long_query_time | 10 |
| low_priority_updates | OFF |
| lower_case_file_system | OFF |
| lower_case_table_names | 0 |
| max_allowed_packet | 16776192 |
| max_binlog_cache_size | 4294967295 |
| max_binlog_size | 1073741824 |
| max_connections | 450 |
| max_connect_errors | 10 |
| max_delayed_threads | 20 |
| max_insert_delayed_threads | 20 |
| max_heap_table_size | 16777216 |
| max_join_size | 4294967295 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_user_connections | 0 |
| max_tmp_tables | 32 |
| max_write_lock_count | 4294967295 |
| myisam_max_extra_sort_file_size | 268435456 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_repair_threads | 1 |
| myisam_recover_options | OFF |
| myisam_sort_buffer_size | 67108864 |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| open_files_limit | 2508 |
| pid_file | /var/lib/mysql/vacuum.pid |
| log_error | |
| port | 3306 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_size | 33554432 |
| query_cache_type | ON |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 2048 |
| read_buffer_size | 1044480 |
| read_only | OFF |
| read_rnd_buffer_size | 262144 |
| rpl_recovery_rank | 0 |
| server_id | 0 |
| slave_net_timeout | 3600 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slow_launch_time | 2 |
| socket | /var/lib/mysql/mysql.sock |
| sort_buffer_size | 2097144 |
| sql_mode | 0 |
| table_cache | 1024 |
| table_type | MYISAM |
| thread_cache_size | 8 |
| thread_stack | 126976 |
| tx_isolation | REPEATABLE-READ |
| timezone | EDT |
| tmp_table_size | 33554432 |
| tmpdir | /tmp/ |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| version | 4.0.20-standard |
| version_comment | Official MySQL RPM |
| version_compile_os | pc-linux |
| wait_timeout | 14400

7. your mysql extended-status output which shows entire server's mysql activity status (preferably with a day or 2 worth of mysql uptime for better indication of real activity) either:

+--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Aborted_clients | 2287 |
| Aborted_connects | 66 |
| Bytes_received | 2282457398 |
| Bytes_sent | 2089106685 |
| Com_admin_commands | 0 |
| Com_alter_table | 0 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 28827218 |
| Com_change_master | 0 |
| Com_check | 0 |
| Com_commit | 0 |
| Com_create_db | 1 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 17 |
| Com_delete | 2264603 |
| Com_delete_multi | 0 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 0 |
| Com_flush | 0 |
| Com_grant | 0 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_insert | 2732104 |
| Com_insert_select | 76835 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 215 |
| Com_optimize | 0 |
| Com_purge | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 528220 |
| Com_replace_select | 1725 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 183196942 |
| Com_set_option | 8362 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 0 |
| Com_show_create | 8362 |
| Com_show_databases | 13 |
| Com_show_fields | 8362 |
| Com_show_grants | 0 |
| Com_show_keys | 0 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 1 |
| Com_show_innodb_status | 0 |
| Com_show_tables | 8622 |
| Com_show_variables | 1 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 215 |
| Com_update | 61869280 |
| Connections | 28827249 |
| Created_tmp_disk_tables | 28000 |
| Created_tmp_tables | 13476253 |
| Created_tmp_files | 2770 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Delayed_errors | 0 |
| Flush_commands | 1 |
| Handler_commit | 0 |
| Handler_delete | 3583596 |
| Handler_read_first | 16050411 |
| Handler_read_key | 2543608738 |
| Handler_read_next | 3448132033 |
| Handler_read_prev | 217591321 |
| Handler_read_rnd | 2609228050 |
| Handler_read_rnd_next | 2962626191 |
| Handler_rollback | 0 |
| Handler_update | 101744696 |
| Handler_write | 2338562772 |
| Key_blocks_used | 15586 |
| Key_read_requests | 3854444022 |
| Key_reads | 75487101 |
| Key_write_requests | 14626384 |
| Key_writes | 13471524 |
| Max_used_connections | 164 |
| Not_flushed_key_blocks | 0 |
| Not_flushed_delayed_rows | 0 |
| Open_tables | 1024 |
| Open_files | 929 |
| Open_streams | 0 |
| Opened_tables | 1170 |
| Questions | 659398876 |
| Qcache_queries_in_cache | 4018 |
| Qcache_inserts | 182383393 |
| Qcache_hits | 351041742 |
| Qcache_lowmem_prunes | 646851 |
| Qcache_not_cached | 813549 |
| Qcache_free_memory | 26581952 |
| Qcache_free_blocks | 2270 |
| Qcache_total_blocks | 10399 |
| Rpl_status | NULL |
| Select_full_join | 27967 |
| Select_full_range_join | 0 |
| Select_range | 32818630 |
| Select_range_check | 0 |
| Select_scan | 28696275 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 28007 |
| Slow_queries | 37441 |
| Sort_merge_passes | 1385 |
| Sort_range | 23508308 |
| Sort_rows | 3278935229 |
| Sort_scan | 14885125 |
| Table_locks_immediate | 332490161 |
| Table_locks_waited | 1241618 |
| Threads_cached | 6 |
| Threads_created | 169368 |
| Threads_connected | 2 |
| Threads_running | 1 |
| Uptime | 17299937 |
+--------------------------+------------+


8. is your vB the only thing on the server? or other scripts & sites which utilise php and mysql?

Mostly. We have a photopost installation but it is unused.

9. how many average and max concurrent users on your vB forum ? and what your cookie timeout is ?
maximum concurrent users = 500, average = 300, low = 70; cookie timeout = 900

10. create a file named phpinfo.php and place this code in it and post the url/link to it from your web site

http://www.tribemagazine.com/phpinfo.php

11. if you run Apache and you have your own dedicated server or access to your httpd.conf (apache configuration file) can you post the values you have set for the following :

KeepAlive = On
MaxKeepAliveRequests = MaxKeepAliveRequests 100
KeepAliveTimeout = 15
MinSpareServers = 5
MaxSpareServers = 20
StartServers = 8
MaxClients = 150

12. what version of vB are you running ? 2.3.6

13. check to see if any files i.e. apache log files are hitting 2GB or 4GB max file size limits i.e. see if you have max file size exceeded messages in apache error log

/var/log/httpd/tribemagazine.com-access.log.1
/var/log/httpd/tribemagazine.com-access.log.2

rotated out regularly

[eva2000]

dual tuallies are still awesome web servers so keep it as web server and a dual AMD64 opteron for database serving

what size are these ?

var/log/httpd/tribemagazine.com-access.log.1
/var/log/httpd/tribemagazine.com-access.log.2

[tribedude]

both at aound 1.9 gig

[tribedude]

George is my existing configuration enough to run 2 boards on it? The other board's traffic probably won't be very big for the first year...

Or is getting anoher server and running a 2 server setup the way to go for our existing big board and another smaller one...

[Bulent Tekcan]

Both 1.9 GB huhhhhh....

Stop your apache delete 2 files and start again...Because these files size very big...

Cheers

[eva2000]

George is my existing configuration enough to run 2 boards on it? The other board's traffic probably won't be very big for the first year...

Or is getting anoher server and running a 2 server setup the way to go for our existing big board and another smaller one...
adding a 2nd server for dedicated mysql database will improve your forum performance alot and should be enough to handle 2 boards for a while... i'd get 3-4GB of ram for database server and come back here to get my.cnf tweaked so the memory buffers get enough memory allocated to take advantage of 3-4GB of ram :)

[tribedude]

How does this one look for a database server for this application?

2 x Opteron 246
MSI K8T Master2-FAR (Dual CPU) motherboard
4 Gig ECC Reg DDR 333
ATI 9250 128M video
Adaptec SCSI RAID 2120S controller
4 x 36 gig seagate 15k scsi drives
SC822R-400RC chassis (2U redundant power supply, 6 SCA hot swap bays)

[eva2000]

yup looks fine to me :)

[AlfonsoC]

How does this one look for a database server for this application?

2 x Opteron 246
MSI K8T Master2-FAR (Dual CPU) motherboard
4 Gig ECC Reg DDR 333
ATI 9250 128M video
Adaptec SCSI RAID 2120S controller
4 x 36 gig seagate 15k scsi drives
SC822R-400RC chassis (2U redundant power supply, 6 SCA hot swap bays)

Have 4 machines very similar to your configuration (well... they wear a Tyan Thunkder K8W 2885 boards).

Basically 3 are dual 246 and one is a dual 250 and I'll assure you that roll over Xeons (is not a matter of CPU Mhz - is a matter of memory bandwidth). Those machines are kings when Mem bw is related. One went up to 8Gb RAM some weeks ago.

BTW: With 246 and upper processors you can go with DDR400 memory. DB servers love memory bandwidth and you can get an extra upping to DDR400 from 333.

One important thing: AVOID at all costs ATI video cards connected on ANY motherboard who uses AMD8000 chipset. There's some incompatibility between them and they spank themselves. Better to use a cheaper nvidia video card (All Opteron servers use a cheaper nvidia 5200 chip who is 'underclocked' - to run a console you will be fine with anything).

Switch from Adaptec 2120S who is a plan PCI 32Bits to a PCI-X SCSI controller (29320 model for example). You do not want to spend extra time waiting for data coming from a 32Bit bus. If possible switch to a PCI-X and connect to a 133/100Mhz PCI-X slot. Which Kind of disk layout plan to use? RAID-5 or RAID 1+0?

And the most important... which OS plan to use? I've used Gentoo on all 4 machines and all 4 are using 64Bit OS and 64Bit compiles of the apps (including Mysql).

[tribedude]

Have 4 machines very similar to your configuration (well... they wear a Tyan Thunkder K8W 2885 boards).

One important thing: AVOID at all costs ATI video cards connected on ANY motherboard who uses AMD8000 chipset. There's some incompatibility between them and they spank themselves. Better to use a cheaper nvidia video card (All Opteron servers use a cheaper nvidia 5200 chip who is 'underclocked' - to run a console you will be fine with anything).

Switch from Adaptec 2120S who is a plan PCI 32Bits to a PCI-X SCSI controller (29320 model for example). You do not want to spend extra time waiting for data coming from a 32Bit bus. If possible switch to a PCI-X and connect to a 133/100Mhz PCI-X slot. Which Kind of disk layout plan to use? RAID-5 or RAID 1+0?

And the most important... which OS plan to use? I've used Gentoo on all 4 machines and all 4 are using 64Bit OS and 64Bit compiles of the apps (including Mysql).


Don't these Tyan Server boards have • ATI® RAGE™ XL PCI graphics controllers built n to them anyway? Will that cause the problem you specify above^.

We will probably go with RAID 5. As for the OS we are currently using Fedora but i am open to suggestions for the new server box.

[tribedude]

I have just got another configuration from my system supplier:

TYAN S2881G2NR Thunder K8SR Main Board
4X 1G ECC Reg.DDR400 RAM(Required Opteron 246 & Higher)
Adaptec 2130SLP SCSI Raid Controller
4X Seagate U320 15K rpm 36G SCSI Hard Drives
CDROM Drive
Supermicro SC832S-R760 3U Chassis
2x Fans
2 x Opteron 246 (which I already have ... purchased on Ebay)

Do I need a dual channel PCI-X SCSI card or is the single channel ^ adequate...

[AlfonsoC]

Don't these Tyan Server boards have • ATI® RAGE™ XL PCI graphics controllers built n to them anyway? Will that cause the problem you specify above^.

We will probably go with RAID 5. As for the OS we are currently using Fedora but i am open to suggestions for the new server box.


Yep... anything over a 7xxx chip on AMD8000 is like playing with an armed bomb... better to not mess with it. Rage XL PCI is fine... K8W is the Workstation board, K8S is the server... slightly different but the same base.

Never liked RAID5 for such environment... if you have enough disks you can get more speed with RAID1+0 and the proper Fs.

For 4 disks, with a single channel board is fine, unless you plan to put more disks on a external cabinet.

[tribedude]

Someone is telling me that adaptec doesn't have drivers for that card for Fedora which is what we will be using and suggested an LSI megaraid card instead. I have never heard of LSI before now... Any good?

[f0urtyfive]

I would suggest CentOS, its a free distribution that immitates RHEL. BTW, Are you coloing your own equipment? If your afraid about the growth of the second forum your starting, you can eventually just add a third server and use it as a dedicated mysql server for the second server. Then if that isnt enough you can add a second webserver too. Oh how fun clustering is :)

[eva2000]

Someone is telling me that adaptec doesn't have drivers for that card for Fedora which is what we will be using and suggested an LSI megaraid card instead. I have never heard of LSI before now... Any good?
yup LSI make excellent controllers especially for linux unlike adaptec which can have issues

[tribedude]

What about RAID flavours? What is best and why? How many drives... Currently not using it on existing system will on the new machine.

Does the webserver need raid or just the database server?

[eva2000]

yeah LSI Logic and 3ware both make good linux raid controllers. Web server won't need raid but depends on what you want and how important the web server data is

[AlfonsoC]

What about RAID flavours? What is best and why? How many drives... Currently not using it on existing system will on the new machine.

Does the webserver need raid or just the database server?

For the database server, don't skimp resources and go for a RAID of 4 disks on a RAID 1+0 configuration I wouldn't go with RAID5 unless the XOR engine of the controller is enough powerful, some RAID controllers lose steam when is related to heavy writes on RAID5 volume.

And do yourself a favor. If use a controller with cache, choose one with a BBU to keep a backup of the data in the cache memory.

For a web server, you can go with a RAID1 without problems and this could be done by software with a volume manager, the file read speed you need for a webserver is provided by the buffering of the file system.