any optimization suggestions :)

[webmasterii]

Hi There, Eva and others :)

My web server used to crash twice a day till i changed the ram from 512 to 2 gig, it is ok now, but it still slows down alot in peak time.

I have two peak times a day, and i have no problem at all in other times, here is the required information about the server, please note that this is not a peak time stats.

any suggestion to optimize the server config is appreciated :) ...

and a quick question, generaly speaking: is it better to have one powerful server, or divid the load by two using the new feature in vb3.

1- Dedicated (from 15minuteservers.com)

2 - Pentium-4 2.4 Ghz, 2Gigs Ram, 80 GB IDE , Fedora-CR3
Apache 1.3.33 , DirectAdmin 1.25.4 , Exim 4.50 , MySQL 4.1.10a
Daily Pageviews 150,000 to 180,000
Posts: 1.8 Million posts.
Some features are disabled or hidden (For performance reasons):
- Online users, emails, calendar, new posts, images, visit counts(delayed) , and most of the features that affects performance.


3- No

4- No Idea ..

5- top - 19:16:24 up 56 days, 7:49, 1 user, load average: 0.21, 0.29, 0.27
Tasks: 230 total, 1 running, 229 sleeping, 0 stopped, 0 zombie
Cpu(s): 38.0% us, 3.6% sy, 0.3% ni, 54.8% id, 2.8% wa, 0.2% hi, 0.3% si
Mem: 2074164k total, 1876656k used, 197508k free, 76612k buffers
Swap: 2048276k total, 284900k used, 1763376k free, 1296592k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7499 root 16 0 2632 1016 700 R 3.9 0.0 0:00.02 top
6369 apache 15 0 29848 10m 2144 S 1.9 0.5 0:07.34 httpd
7497 root 16 0 5712 1680 1372 S 1.9 0.1 0:00.01 sshd
1 root 16 0 1840 496 472 S 0.0 0.0 0:51.63 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.43 ksoftirqd/0
4 root 5 -10 0 0 0 S 0.0 0.0 2:48.20 events/0
5 root 7 -10 0 0 0 S 0.0 0.0 0:00.01 khelper
17 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
100 root 5 -10 0 0 0 S 0.0 0.0 7:20.48 kblockd/0
169 root 9 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
168 root 15 0 0 0 0 S 0.0 0.0 158:30.29 kswapd0
108 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
260 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
467 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 ata/0
480 root 25 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0
481 root 25 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1
489 root 15 0 0 0 0 S 0.0 0.0 1:05.00 kjournald
1456 root 6 -10 3440 388 388 S 0.0 0.0 0:00.02 udevd
2094 root 15 0 0 0 0 S 0.0 0.0 0:00.03 kjournald
2095 root 15 0 0 0 0 S 0.0 0.0 0:46.51 kjournald
2096 root 15 0 0 0 0 S 0.0 0.0 6:56.31 kjournald
2097 root 15 0 0 0 0 S 0.0 0.0 0:09.37 kjournald
2098 root 15 0 0 0 0 S 0.0 0.0 3:23.29 kjournald
3063 root 16 0 3188 544 476 S 0.0 0.0 1:08.75 syslogd
3067 root 16 0 2936 460 404 S 0.0 0.0 0:00.02 klogd
3121 root 16 0 5708 860 836 S 0.0 0.0 0:02.38 rpc.idmapd
3277 root 16 0 2912 556 480 S 0.0 0.0 3:22.11 nifd
3293 root 20 0 2524 444 444 S 0.0 0.0 0:00.00 acpid
3360 root 15 0 4612 1200 1088 S 0.0 0.1 2:09.33 sshd
3392 mail 15 0 8104 1536 1444 S 0.0 0.1 0:25.22 exim
3463 ftp 16 0 3396 736 624 S 0.0 0.0 0:10.88 proftpd
3473 root 16 0 3248 436 436 S 0.0 0.0 0:00.00 vm-pop3d
3492 root 16 0 4360 712 656 S 0.0 0.0 0:10.14 crond
3545 daemon 16 0 2516 600 564 S 0.0 0.0 0:00.73 atd
3561 dbus 16 0 3384 592 592 S 0.0 0.0 0:00.04 dbus-daemon-1
3576 root 16 0 7696 1668 724 S 0.0 0.1 1:29.70 hald
3627 root 18 0 2032 348 348 S 0.0 0.0 0:00.00 mingetty
3630 root 19 0 2360 348 348 S 0.0 0.0 0:00.00 mingetty
3631 root 18 0 2520 348 348 S 0.0 0.0 0:00.00 mingetty
3632 root 18 0 3232 348 348 S 0.0 0.0 0:00.00 mingetty

6- [mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-name-resolve
skip-innodb
max_connections = 650
key_buffer = 32M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 128
wait_timeout = 900
connect_timeout = 10
tmp_table_size = 64M
max_allowed_packet = 16M
max_connect_errors = 10
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 2M
query_cache_size = 96M
query_cache_type = 1
query_prealloc_size = 16384
query_alloc_block_size = 16384

[mysql.server]
user=mysql
basedir=/var/lib

[mysqld_safe]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M


7- +--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Aborted_clients | 139670 |
| Aborted_connects | 17189 |
| Binlog_cache_disk_use | 0 |
| Binlog_cache_use | 0 |
| Bytes_received | 3592538822 |
| Bytes_sent | 3512067873 |
| Com_admin_commands | 4 |
| Com_alter_db | 0 |
| Com_alter_table | 4 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 7846314 |
| Com_change_master | 0 |
| Com_check | 0 |
| Com_checksum | 0 |
| Com_commit | 0 |
| Com_create_db | 2 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 35 |
| Com_dealloc_sql | 0 |
| Com_delete | 820817 |
| Com_delete_multi | 0 |
| Com_do | 0 |
| Com_drop_db | 2 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 0 |
| Com_drop_user | 0 |
| Com_execute_sql | 0 |
| Com_flush | 6 |
| Com_grant | 4 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_help | 0 |
| Com_insert | 5055745 |
| Com_insert_select | 136068 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 6 |
| Com_optimize | 0 |
| Com_preload_keys | 0 |
| Com_prepare_sql | 0 |
| Com_purge | 0 |
| Com_purge_before_date | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 574635 |
| Com_replace_select | 0 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 36271217 |
| Com_set_option | 401 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 0 |
| Com_show_charsets | 0 |
| Com_show_collations | 0 |
| Com_show_column_types | 0 |
| Com_show_create_db | 0 |
| Com_show_create_table | 394 |
| Com_show_databases | 529 |
| Com_show_errors | 0 |
| Com_show_fields | 395 |
| Com_show_grants | 0 |
| Com_show_innodb_status | 0 |
| Com_show_keys | 0 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_privileges | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 1 |
| Com_show_storage_engines | 0 |
| Com_show_tables | 766 |
| Com_show_variables | 171 |
| Com_show_warnings | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 7 |
| Com_update | 9574977 |
| Com_update_multi | 0 |
| Connections | 7891180 |
| Created_tmp_disk_tables | 50436 |
| Created_tmp_files | 19215 |
| Created_tmp_tables | 407044 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Flush_commands | 1 |
| Handler_commit | 0 |
| Handler_delete | 2172017 |
| Handler_discover | 0 |
| Handler_read_first | 4341235 |
| Handler_read_key | 1042040870 |
| Handler_read_next | 3640144465 |
| Handler_read_prev | 91455243 |
| Handler_read_rnd | 80305667 |
| Handler_read_rnd_next | 3436239062 |
| Handler_rollback | 0 |
| Handler_update | 12138237 |
| Handler_write | 31721612 |
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 0 |
| Key_blocks_used | 28995 |
| Key_read_requests | 4270578688 |
| Key_reads | 31892187 |
| Key_write_requests | 21091301 |
| Key_writes | 14781955 |
| Max_used_connections | 651 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 1910 |
| Open_streams | 0 |
| Open_tables | 1800 |
| Opened_tables | 29276 |
| Qcache_free_blocks | 15681 |
| Qcache_free_memory | 61118976 |
| Qcache_hits | 43766411 |
| Qcache_inserts | 36015113 |
| Qcache_lowmem_prunes | 77502 |
| Qcache_not_cached | 256101 |
| Qcache_queries_in_cache | 24464 |
| Qcache_total_blocks | 65185 |
| Questions | 111834925 |
| Rpl_status | NULL |
| Select_full_join | 2209 |
| Select_full_range_join | 0 |
| Select_range | 11449190 |
| Select_range_check | 0 |
| Select_scan | 5609029 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 9675 |
| Slow_queries | 135362 |
| Sort_merge_passes | 9637 |
| Sort_range | 10132761 |
| Sort_rows | 291206874 |
| Sort_scan | 217400 |
| Table_locks_immediate | 95702220 |
| Table_locks_waited | 779580 |
| Threads_cached | 125 |
| Threads_connected | 3 |
| Threads_created | 26090 |
| Threads_running | 1 |
| Uptime | 3705896 |
+--------------------------+------------+


8- Only Vbulletin, there is another test Vbulletin in the server but hardly used.

9- 400 – 800 users, Timeout is 900

10 - http://www.omania.net/phpinfo.php (http://www.omania.net/phpinfo.php)

11- Timeout 300
KeepAlive On
MaxKeepAliveRequests 500
KeepAliveTimeout 5
MinSpareServers 5
MaxSpareServers 20
StartServers 8
MaxClients 450
MaxRequestsPerChild 1000

12- VB 3.1

13- Nothing

[webmasterii]

I have just installed Eaccelerator 0.9.3 ..
was easy thanks to Erwin ..
I understand it will help when peak time comes :) ..

[eva2000]

single P4 2.4GHZ cpu server is under powered to handle your peak times it seems as you're handling 651+ mysql concurrent connections to the mysql server which hit your preset max_connection limit of 650 already suggesting you may generate even more connections due to forum activity

that kind of load is in the realm of really 2x dual cpu servers to handle the load!

But try the following first and see how it goes

1. Upgrade to MySQL 4.1.14/15 check if ensim supports it

2. Edit /etc/my.cnf and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards


[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-name-resolve
safe-show-database
skip-innodb
max_connections = 650
key_buffer = 32M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 384
wait_timeout = 180
connect_timeout = 10
tmp_table_size = 128M
max_allowed_packet = 64M
max_connect_errors = 10
thread_concurrency = 2
query_cache_limit = 1M
query_cache_size = 64M
query_cache_type = 1
query_prealloc_size = 16384
query_alloc_block_size = 16384

[mysql.server]
user=mysql
basedir=/var/lib

[mysqld_safe]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 256M
sort_buffer = 256M
read_buffer = 128M
write_buffer = 128M



3. change httpd.conf from

Timeout 300
KeepAlive On
MaxKeepAliveRequests 500
KeepAliveTimeout 5
MinSpareServers 5
MaxSpareServers 20
StartServers 8
MaxClients 450
MaxRequestsPerChild 1000

to

Timeout 300
KeepAlive Off
MaxKeepAliveRequests 120
KeepAliveTimeout 5
MinSpareServers 5
MaxSpareServers 15
StartServers 8
MaxClients 200
MaxRequestsPerChild 500

restart apache

try with keepalive off and on and see which is better

[webmasterii]

single P4 2.4GHZ cpu server is under powered to handle your peak times it seems as you're handling 651+ mysql concurrent connections to the mysql server which hit your preset max_connection limit of 650 already suggesting you may generate even more connections due to forum activity
that kind of load is in the realm of really 2x dual cpu servers to handle the load!

You are right, but I can not afford the costs :(
I did the advised changes .. Thanks ..


is there any tool to measure the server load all the day, as i can not be online at the peak time, and would like to see some kind of a log of the server load during the day , or the week.

Cheers :cool:

[eva2000]

well there's cacti but it's quite a bit of work to setup http://www.cacti.net/