PDA

View Full Version : Servers tweak request + apache crash...


lewekleonek
Thu 10th Mar '05, 12:16pm
Hi,
We have tweaked our server as much as we could but we still observe some problems. Apache crashes, general slowdowns of the board etc. Can someone please provide us some tips on how to improve the situation?

1. Two Dedicated servers:
web server: only one VBulletin forum + some other small php apps (phpAdsNew, phplist).
database server: one mysql instance serving the above mentioned web server.
2. Web server specs:
CPU: Dual Intel(R) Xeon(TM) CPU 2.40GHz stepping 09/HyperThreading enabled = + 2 virtual CPUs
RAM: 1024M
HDD: SCSI - U320 RAID mirror configuration - 2x72GB = 72GB storage space
Linux Distro: Debian 3.1
Web Server: Apache 1.3.33
PHP: 4.3.10
Turck-MMCache: 2.4.6

Database Server Specs:
CPU: Dual Intel(R) Xeon(TM) CPU 2.66GHz stepping 09
RAM:
HDD: SCSI - U160 36GB drive
Linux Distro: Debian 3.1
MySQL: 4.0.23

3. InnoDB disabled.
4. Debian package (pre-compiled): mysql-server 4.0.23-7
5. Top Stats. (forum site is fairly busy - 720 users)
Web Server:

top - 11:15:26 up 10 days, 1:28, 1 user, load average: 3.49, 5.67, 5.67
Tasks: 478 total, 1 running, 476 sleeping, 0 stopped, 1 zombie
Cpu0 : 62.0% us, 5.6% sy, 0.0% ni, 26.9% id, 1.3% wa, 1.3% hi, 3.0% si
Cpu1 : 56.6% us, 6.2% sy, 0.0% ni, 35.9% id, 0.7% wa, 0.0% hi, 0.7% si
Cpu2 : 55.7% us, 6.6% sy, 0.0% ni, 32.8% id, 1.3% wa, 1.3% hi, 2.3% si
Cpu3 : 51.3% us, 6.9% sy, 0.0% ni, 40.1% id, 1.0% wa, 0.0% hi, 0.7% si
Mem: 1033024k total, 885448k used, 147576k free, 17464k buffers
Swap: 995988k total, 170904k used, 825084k free, 143540k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
24328 sadmin 15 0 34768 7736 4860 S 17.1 0.7 0:02.33 apache
24617 sadmin 15 0 34468 7356 4780 S 15.8 0.7 0:04.98 apache
24656 sadmin 16 0 35292 7848 4448 S 13.5 0.8 0:00.41 apache
24281 sadmin 15 0 35236 8216 4872 S 11.8 0.8 0:02.74 apache
24410 sadmin 15 0 34632 7744 4980 S 9.2 0.7 0:04.46 apache
24660 sadmin 15 0 34596 7156 4452 S 9.2 0.7 0:00.28 apache
24409 sadmin 16 0 33828 6200 4268 S 8.6 0.6 0:01.93 apache
23158 sadmin 15 0 34588 7660 4944 S 7.9 0.7 0:07.35 apache
24377 sadmin 15 0 34536 8088 5428 S 7.9 0.8 0:06.47 apache
24472 sadmin 15 0 34684 7388 4596 S 7.9 0.7 0:03.89 apache
24498 sadmin 15 0 34436 7332 4788 S 7.9 0.7 0:04.29 apache
24514 sadmin 15 0 34496 7156 4552 S 7.2 0.7 0:02.42 apache
23145 sadmin 15 0 34556 7584 4916 S 6.6 0.7 0:06.94 apache
23287 sadmin 15 0 34424 7096 4572 S 5.9 0.7 0:03.28 apache
24323 sadmin 15 0 34444 8256 5684 S 5.6 0.8 0:02.67 apache
24556 sadmin 15 0 34748 7812 4936 S 5.3 0.8 0:02.96 apache
23409 sadmin 15 0 34320 6552 4612 S 4.6 0.6 0:02.73 apache
24278 sadmin 15 0 34448 7940 5376 S 4.6 0.8 0:04.49 apache
23535 sadmin 15 0 33924 6324 4480 S 4.3 0.6 0:00.66 apache
24509 sadmin 15 0 34584 7664 4972 S 4.3 0.7 0:03.49 apache
24535 sadmin 15 0 34404 7320 4808 S 4.3 0.7 0:03.21 apache
24662 sadmin 17 0 34924 7464 4432 S 4.3 0.7 0:00.13 apache
24568 sadmin 15 0 34576 7240 4556 S 3.3 0.7 0:01.83 apache
23898 sadmin 15 0 34512 7232 4672 S 3.0 0.7 0:07.77 apache
24508 sadmin 15 0 34740 7780 4896 S 3.0 0.8 0:05.27 apache
24519 sadmin 15 0 34552 7336 4676 S 3.0 0.7 0:02.83 apache
23190 sadmin 16 0 34352 7468 5016 S 2.6 0.7 0:04.08 apache
23578 sadmin 15 0 34468 7876 5360 S 2.6 0.8 0:06.77 apache
24279 sadmin 15 0 33832 5948 4020 S 2.6 0.6 0:04.45 apache
24301 sadmin 15 0 34432 7484 4944 S 2.6 0.7 0:01.83 apache
24518 sadmin 15 0 34564 7260 4588 S 2.6 0.7 0:01.43 apache
24566 sadmin 15 0 34568 7284 4608 S 2.6 0.7 0:01.68 apache
23019 sadmin 15 0 34568 7588 4912 S 2.3 0.7 0:10.63 apache
23058 sadmin 15 0 34624 7532 4812 S 2.0 0.7 0:06.19 apache
24298 sadmin 15 0 34520 7568 4920 S 2.0 0.7 0:05.04 apache
23153 sadmin 15 0 34496 7240 4636 S 1.6 0.7 0:03.48 apache
23271 sadmin 15 0 34588 7568 4872 S 1.6 0.7 0:04.59 apache
23373 sadmin 15 0 34420 7144 4616 S 1.6 0.7 0:02.32 apache
24231 sadmin 15 0 34452 7480 4916 S 1.6 0.7 0:04.55 apache
24324 sadmin 15 0 34476 7452 4868 S 1.6 0.7 0:02.99 apache
24333 sadmin 15 0 34396 7160 4656 S 1.6 0.7 0:01.48 apache
24345 sadmin 15 0 34584 7376 4684 S 1.6 0.7 0:06.66 apache
24380 sadmin 16 0 34532 7456 4812 S 1.6 0.7 0:05.28 apache
24653 sadmin 15 0 33840 5708 3840 S 1.6 0.6 0:00.05 apache
23536 sadmin 16 0 34340 6968 4584 S 1.3 0.7 0:00.41 apache
23873 sadmin 15 0 34528 7432 4796 S 1.3 0.7 0:10.23 apache
24149 sadmin 16 0 34768 8104 5188 S 1.3 0.8 0:08.41 apache
24168 sadmin 15 0 34660 7868 5064 S 1.3 0.8 0:10.02 apache
24382 sadmin 16 0 34420 7572 5044 S 1.3 0.7 0:03.12 apache
24511 sadmin 15 0 34500 7200 4592 S 1.3 0.7 0:01.33 apache
24567 sadmin 15 0 34460 7328 4756 S 1.3 0.7 0:01.74 apache
24658 sadmin 16 0 33800 5692 3832 S 1.3 0.6 0:00.04 apache
24659 sadmin 15 0 33800 5700 3840 S 1.3 0.6 0:00.04 apache
24661 sadmin 15 0 33800 5692 3832 S 1.3 0.6 0:00.04 apache
24219 sadmin 16 0 34608 7856 5140 S 1.0 0.8 0:04.26 apache
24304 sadmin 16 0 34456 7236 4672 S 1.0 0.7 0:04.18 apache
24376 sadmin 15 0 34576 7376 4692 S 1.0 0.7 0:03.71 apache
24496 sadmin 15 0 33832 5788 3860 S 1.0 0.6 0:04.83 apache
24499 sadmin 15 0 34448 7148 4592 S 1.0 0.7 0:01.74 apache
24570 sadmin 16 0 34528 7428 4792 S 1.0 0.7 0:00.79 apache
24652 sadmin 16 0 2328 1308 844 R 1.0 0.1 0:00.21 top
23208 sadmin 15 0 34440 7620 5088 S 0.7 0.7 0:04.52 apache
23338 sadmin 16 0 34772 8008 5108 S 0.7 0.8 0:03.26 apache
23479 sadmin 16 0 34640 7292 4604 S 0.7 0.7 0:01.29 apache



Database Server:

top - 11:03:05 up 2 days, 20:58, 1 user, load average: 0.30, 0.43, 0.43
Tasks: 52 total, 1 running, 51 sleeping, 0 stopped, 0 zombie
Cpu0 : 7.4% us, 6.0% sy, 0.0% ni, 83.6% id, 0.0% wa, 0.7% hi, 2.3% si
Cpu1 : 6.4% us, 6.0% sy, 0.0% ni, 84.3% id, 0.0% wa, 1.0% hi, 2.3% si
Mem: 2075132k total, 1684180k used, 390952k free, 29172k buffers
Swap: 1951856k total, 0k used, 1951856k free, 1148996k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26832 mysql 15 0 409m 300m 2736 S 30.0 14.8 614:21.85 mysqld
19704 admin 16 0 2068 1064 844 R 0.3 0.1 0:00.04 top
1 root 16 0 1500 516 456 S 0.0 0.0 0:00.46 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.06 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.54 ksoftirqd/0
4 root RT 0 0 0 0 S 0.0 0.0 0:00.05 migration/1
5 root 34 19 0 0 0 S 0.0 0.0 0:00.61 ksoftirqd/1
6 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/0
7 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/1
8 root 5 -10 0 0 0 S 0.0 0.0 0:00.03 khelper
21 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
103 root 5 -10 0 0 0 S 0.0 0.0 0:00.13 kblockd/0
104 root 5 -10 0 0 0 S 0.0 0.0 0:00.01 kblockd/1
175 root 15 0 0 0 0 S 0.0 0.0 0:10.71 pdflush
176 root 15 0 0 0 0 S 0.0 0.0 0:01.60 pdflush
178 root 14 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
177 root 20 0 0 0 0 S 0.0 0.0 0:00.46 kswapd0
179 root 14 -10 0 0 0 S 0.0 0.0 0:00.00 aio/1
772 root 25 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
817 root 17 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0
818 root 15 0 0 0 0 S 0.0 0.0 0:00.00 ahc_dv_0
836 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 exec-osm/0
837 root 7 -10 0 0 0 S 0.0 0.0 0:00.00 exec-osm/1
841 root 15 0 0 0 0 S 0.0 0.0 0:00.28 kirqd
974 root 19 0 0 0 0 S 0.0 0.0 0:00.00 khubd
1193 root 16 0 1560 648 528 S 0.0 0.0 0:00.68 syslogd
1196 root 20 0 1508 472 404 S 0.0 0.0 0:00.00 klogd
1204 root 15 0 1540 532 464 S 0.0 0.0 0:00.00 inetd
1215 spong 21 0 6696 5324 1792 S 0.0 0.3 0:09.34 spong-client
1233 root 16 0 3468 1540 1300 S 0.0 0.1 0:00.19 sshd
1239 daemon 16 0 1684 640 556 S 0.0 0.0 0:00.00 atd
1242 root 16 0 1764 824 680 S 0.0 0.0 0:00.09 cron
1257 root 16 0 1504 488 420 S 0.0 0.0 0:00.00 getty
1259 root 16 0 1504 488 420 S 0.0 0.0 0:00.00 getty
1260 root 16 0 1504 488 420 S 0.0 0.0 0:00.00 getty
1261 root 17 0 1504 488 420 S 0.0 0.0 0:00.00 getty
1262 root 16 0 1504 488 420 S 0.0 0.0 0:00.00 getty
1263 root 16 0 1504 488 420 S 0.0 0.0 0:00.00 getty
26796 root 25 0 2320 1104 904 S 0.0 0.1 0:00.00 mysqld_safe
26833 root 24 0 1488 500 436 S 0.0 0.0 0:00.00 logger
845 root 16 0 13276 5684 3992 S 0.0 0.3 0:00.12 apache
851 www-data 16 0 13276 5728 3992 S 0.0 0.3 0:00.00 apache
852 www-data 16 0 13276 5784 4036 S 0.0 0.3 0:00.00 apache
853 www-data 16 0 13276 5788 4040 S 0.0 0.3 0:00.00 apache
854 www-data 16 0 13416 5896 4036 S 0.0 0.3 0:00.00 apache
855 www-data 16 0 13276 5784 4036 S 0.0 0.3 0:00.00 apache
23769 www-data 16 0 13276 5732 3996 S 0.0 0.3 0:00.00 apache
23772 www-data 16 0 13276 5784 4036 S 0.0 0.3 0:00.00 apache
14095 www-data 16 0 13276 5700 3996 S 0.0 0.3 0:00.00 apache
19302 root 16 0 15472 2184 1676 S 0.0 0.1 0:00.01 sshd
19366 admin 16 0 15640 2268 1732 S 0.0 0.1 0:00.08 sshd
19367 admin 15 0 2592 1480 1168 S 0.0 0.1 0:00.00 bash


6. MySQL variables:

+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| back_log | 50 |
| basedir | /usr/ |
| bdb_cache_size | 8388600 |
| bdb_log_buffer_size | 768000 |
| bdb_home | /var/lib/mysql/ |
| bdb_max_lock | 10000 |
| bdb_logdir | |
| bdb_shared_data | OFF |
| bdb_tmpdir | /tmp/ |
| bdb_version | Sleepycat Software: Berkeley DB 3.2.9a: (December 18, 2004) |
| binlog_cache_size | 32768 |
| bulk_insert_buffer_size | 8388608 |
| character_set | latin1 |
| character_sets | latin1 big5 cp1251 cp1257 croat czech danish dec8 dos estonia euc_kr gb2312 gbk german1 greek hebrew hp8 hungarian koi8_ru koi8_ukr latin1_de latin2 latin5 sjis swe7 tis620 ujis usa7 win1250 win1251ukr win1251 |
| concurrent_insert | ON |
| connect_timeout | 10 |
| convert_character_set | |
| datadir | /var/lib/mysql/ |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_min_word_len | 4 |
| ft_max_word_len | 254 |
| ft_max_word_len_for_sort | 20 |
| ft_stopword_file | (built-in) |
| have_bdb | YES |
| have_crypt | YES |
| have_innodb | DISABLED |
| have_isam | YES |
| have_raid | YES |
| have_symlink | YES |
| have_openssl | NO |
| have_query_cache | YES |
| init_file | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_buffer_pool_size | 8388608 |
| innodb_data_file_path | |
| innodb_data_home_dir | |
| innodb_file_io_threads | 4 |
| innodb_force_recovery | 0 |
| innodb_thread_concurrency | 8 |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_fast_shutdown | ON |
| innodb_flush_method | |
| innodb_lock_wait_timeout | 50 |
| innodb_log_arch_dir | |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 1048576 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | |
| innodb_mirrored_log_groups | 1 |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_table_locks | ON |
| interactive_timeout | 28800 |
| join_buffer_size | 1044480 |
| key_buffer_size | 67108864 |
| language | /usr/share/mysql/english/ |
| large_files_support | ON |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_update | OFF |
| log_bin | OFF |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_warnings | 1 |
| long_query_time | 10 |
| low_priority_updates | OFF |
| lower_case_file_system | OFF |
| lower_case_table_names | 0 |
| max_allowed_packet | 16776192 |
| max_binlog_cache_size | 4294967295 |
| max_binlog_size | 1073741824 |
| max_connections | 1024 |
| max_connect_errors | 10 |
| max_delayed_threads | 20 |
| max_insert_delayed_threads | 20 |
| max_heap_table_size | 16777216 |
| max_join_size | 4294967295 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_user_connections | 0 |
| max_tmp_tables | 32 |
| max_write_lock_count | 4294967295 |
| myisam_max_extra_sort_file_size | 268435456 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_repair_threads | 1 |
| myisam_recover_options | OFF |
| myisam_sort_buffer_size | 67108864 |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| open_files_limit | 5130 |
| pid_file | /var/run/mysqld/mysqld.pid |
| log_error | |
| port | 3306 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_size | 33554432 |
| query_cache_type | ON |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 2048 |
| read_buffer_size | 1044480 |
| read_only | OFF |
| read_rnd_buffer_size | 262144 |
| rpl_recovery_rank | 0 |
| server_id | 1 |
| slave_net_timeout | 3600 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slow_launch_time | 2 |
| socket | /var/run/mysqld/mysqld.sock |
| sort_buffer_size | 2097144 |
| sql_mode | 0 |
| table_cache | 1500 |
| table_type | MYISAM |
| thread_cache_size | 64 |
| thread_stack | 196608 |
| tx_isolation | REPEATABLE-READ |
| timezone | EST |
| tmp_table_size | 33554432 |
| tmpdir | /tmp/ |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| version | 4.0.23_Debian-7 |
| version_comment | Source distribution |
| version_compile_os | pc-linux-gnu |
| wait_timeout | 3600 |
+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+



7. MySQL extended status:

+--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Aborted_clients | 0 |
| Aborted_connects | 36367 |
| Bytes_received | 1413940863 |
| Bytes_sent | 301023484 |
| Com_admin_commands | 7 |
| Com_alter_table | 0 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 14309576 |
| Com_change_master | 0 |
| Com_check | 268 |
| Com_commit | 0 |
| Com_create_db | 0 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 0 |
| Com_delete | 21874 |
| Com_delete_multi | 0 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 0 |
| Com_flush | 2 |
| Com_grant | 0 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_insert | 744864 |
| Com_insert_select | 1011 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 0 |
| Com_optimize | 0 |
| Com_purge | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 15011 |
| Com_replace_select | 0 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 17778786 |
| Com_set_option | 0 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 3 |
| Com_show_create | 0 |
| Com_show_databases | 11 |
| Com_show_fields | 9 |
| Com_show_grants | 0 |
| Com_show_keys | 0 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_processlist | 18 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 17 |
| Com_show_innodb_status | 0 |
| Com_show_tables | 72 |
| Com_show_variables | 6 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 0 |
| Com_update | 13420311 |
| Connections | 14345364 |
| Created_tmp_disk_tables | 686 |
| Created_tmp_tables | 75838 |
| Created_tmp_files | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Delayed_errors | 0 |
| Flush_commands | 1 |
| Handler_commit | 0 |
| Handler_delete | 236674 |
| Handler_read_first | 5457505 |
| Handler_read_key | 478448013 |
| Handler_read_next | 327663060 |
| Handler_read_prev | 15384962 |
| Handler_read_rnd | 123649694 |
| Handler_read_rnd_next | 882009106 |
| Handler_rollback | 0 |
| Handler_update | 5612576 |
| Handler_write | 95622477 |
| Key_blocks_used | 62345 |
| Key_read_requests | 1146255432 |
| Key_reads | 81512 |
| Key_write_requests | 813116 |
| Key_writes | 748877 |
| Max_used_connections | 1024 |
| Not_flushed_key_blocks | 0 |
| Not_flushed_delayed_rows | 0 |
| Open_tables | 1027 |
| Open_files | 1260 |
| Open_streams | 0 |
| Opened_tables | 1033 |
| Questions | 104508584 |
| Qcache_queries_in_cache | 2015 |
| Qcache_inserts | 17425303 |
| Qcache_hits | 43908239 |
| Qcache_lowmem_prunes | 21716 |
| Qcache_not_cached | 353483 |
| Qcache_free_memory | 20508568 |
| Qcache_free_blocks | 2568 |
| Qcache_total_blocks | 6833 |
| Rpl_status | NULL |
| Select_full_join | 45 |
| Select_full_range_join | 1 |
| Select_range | 727583 |
| Select_range_check | 0 |
| Select_scan | 156517 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 0 |
| Slow_queries | 3 |
| Sort_merge_passes | 0 |
| Sort_range | 728008 |
| Sort_rows | 144794105 |
| Sort_scan | 81221 |
| Table_locks_immediate | 79629062 |
| Table_locks_waited | 418913 |
| Threads_cached | 59 |
| Threads_created | 47103 |
| Threads_connected | 490 |
| Threads_running | 1 |
| Uptime | 207527 |
+--------------------------+------------+



8. only one VBulletin forum + some other small php apps (phpAdsNew, phplist).

9. Average forum users: 650 - 700; max forum users: 1172; cookie time-out: 4800

10. phpinfo() output:

PHP Version => 4.3.10

System => Linux macroweb.webair.com 2.6.10-20051021 #1 SMP Fri Jan 21 20:30:46 EST 2005 i686
Build Date => Feb 10 2005 20:09:55
Configure Command => './configure' '--prefix=/www/php' '--sysconfdir=/etc' '--with-apxs=/www/apache/sbin/apxs' '--with-regex=php' '--with-config-file-path=/www/php/conf' '--with-layout=GNU' '--disable-rpath' '--with-pear=/www/php/share' '--enable-calendar' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' '--enable-track-vars' '--enable-track-vars' '--enable-ftp' '--with-mime-magic=/usr/share/misc/file/magic.mime' '--with-exec-dir=/www/php/libexec' '--with-openssl=/usr' '--with-zlib' '--with-mysql=/usr' '--with-gd=/usr' '--with-jpeg-dir=/usr' '--with-png-dir=/usr' '--with-freetype-dir=/usr' '--enable-gd-native-ttf' '--with-dom' '--with-dom' '--with-dom-xslt' '--with-dom-exslt' '--enable-inline-optimization' '--disable-debug' '--with-imap' '--with-imap-ssl' '--with-kerberos' '--enable-mbstring' '--no-create' '--no-recursion'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /www/php/conf/php.ini
PHP API => 20020918
PHP Extension => 20020429
Zend Extension => 20021010
Debug Build => no
Thread Safety => disabled
Registered PHP Streams => php, http, ftp, https, ftps, compress.zlib


This program makes use of the Zend Scripting Language Engine:
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies
with Turck MMCache v2.4.6, Copyright (c) 2002-2003 TurckSoft, St. Petersburg, by Dmitry Stogov


__________________________________________________ _____________________


Configuration

PHP Core

Directive => Local Value => Master Value
allow_call_time_pass_reference => On => On
allow_url_fopen => On => On
always_populate_raw_post_data => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
asp_tags => Off => Off
auto_append_file => no value => no value
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => no value => no value
default_mimetype => text/html => text/html
define_syslog_variables => Off => Off
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => Off => Off
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => On => On
error_append_string => no value => no value
error_log => /www/logs/phperror.log => /www/logs/phperror.log
error_prepend_string => no value => no value
error_reporting => 2039 => 2039
expose_php => On => On
extension_dir => /www/php/lib/php/20020429 => /www/php/lib/php/20020429
file_uploads => On => On
gpc_order => GPC => GPC
highlight.bg => #FFFFFF => #FFFFFF
highlight.comment => #FF8000 => #FF8000
highlight.default => #0000BB => #0000BB
highlight.html => #000000 => #000000
highlight.keyword => #007700 => #007700
highlight.string => #DD0000 => #DD0000
html_errors => Off => On
ignore_repeated_errors => Off => Off
ignore_repeated_source => Off => Off
ignore_user_abort => Off => Off
implicit_flush => On => Off
include_path => .:/www/php/share => .:/www/php/share
log_errors => On => On
log_errors_max_len => 1024 => 1024
magic_quotes_gpc => On => On
magic_quotes_runtime => Off => Off
magic_quotes_sybase => Off => Off
max_execution_time => 0 => 30
max_input_time => 60 => 60
open_basedir => no value => no value
output_buffering => 0 => no value
output_handler => no value => no value
post_max_size => 16M => 16M
precision => 12 => 12
register_argc_argv => On => On
register_globals => Off => Off
report_memleaks => On => On
safe_mode => Off => Off
safe_mode_exec_dir => no value => no value
safe_mode_gid => Off => Off
safe_mode_include_dir => no value => no value
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
serialize_precision => 100 => 100
short_open_tag => On => On
SMTP => localhost => localhost
smtp_port => 25 => 25
sql.safe_mode => Off => Off
track_errors => Off => Off
unserialize_callback_func => no value => no value
upload_max_filesize => 8M => 8M
upload_tmp_dir => /tmp => /tmp
user_dir => no value => no value
variables_order => EGPCS => EGPCS
xmlrpc_error_number => 0 => 0
xmlrpc_errors => Off => Off
y2k_compliance => On => On

Turck MMCache

MMCache support => enabled
Version => 2.4.6
Caching Enabled => false
Optimizer Enabled => false

Directive => Local Value => Master Value
mmcache.cache_dir => /tmp/mmcache => /tmp/mmcache
mmcache.check_mtime => 1 => 1
mmcache.compress => 1 => 1
mmcache.content => shm_and_disk => shm_and_disk
mmcache.debug => 0 => 0
mmcache.enable => 1 => 1
mmcache.filter => no value => no value
mmcache.keys => shm_and_disk => shm_and_disk
mmcache.optimizer => 1 => 1
mmcache.sessions => shm_and_disk => shm_and_disk
mmcache.shm_max => 0 => 0
mmcache.shm_only => 1 => 1
mmcache.shm_prune_period => 0 => 0
mmcache.shm_size => 16 => 16
mmcache.shm_ttl => 0 => 0

calendar

Calendar support => enabled

ctype

ctype functions => enabled

domxml

DOM/XML => enabled
DOM/XML API Version => 20020815
libxml Version => 20611
HTML Support => enabled
XPath Support => enabled
XPointer Support => enabled
DOM/XSLT => enabled
libxslt Version => 1.1.8
libxslt compiled against libxml Version => 2.6.11
DOM/EXSLT => enabled
libexslt Version => 1.1.8

ftp

FTP support => enabled

gd

GD Support => enabled
GD Version => 2.0 or higher
FreeType Support => enabled
FreeType Linkage => with freetype
GIF Read Support => enabled
GIF Create Support => enabled
JPG Support => enabled
PNG Support => enabled
WBMP Support => enabled

imap

IMAP c-Client Version => 2001
SSL Support => enabled
Kerberos Support => enabled

mbstring

Multibyte Support => enabled
Japanese support => enabled
Simplified chinese support => enabled
Traditional chinese support => enabled
Korean support => enabled
Russian support => enabled
Multibyte (japanese) regex support => enabled

mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.

Directive => Local Value => Master Value
mbstring.detect_order => no value => no value
mbstring.encoding_translation => Off => Off
mbstring.func_overload => 0 => 0
mbstring.http_input => pass => pass
mbstring.http_output => pass => pass
mbstring.internal_encoding => ISO-8859-1 => no value
mbstring.language => neutral => neutral
mbstring.substitute_character => no value => no value

mime_magic

mime_magic support => enabled

Directive => Local Value => Master Value
mime_magic.magicfile => /usr/share/misc/file/magic.mime => /usr/share/misc/file/magic.mime

mysql

MySQL Support => enabled
Active Persistent Links => 0
Active Links => 0
Client API version => 4.0.23
MYSQL_MODULE_TYPE => external
MYSQL_SOCKET => /var/run/mysqld/mysqld.sock
MYSQL_INCLUDE => -I/usr/include/mysql
MYSQL_LIBS => -L/usr/lib -lmysqlclient

Directive => Local Value => Master Value
mysql.allow_persistent => Off => Off
mysql.connect_timeout => 20 => 20
mysql.default_host => no value => no value
mysql.default_password => no value => no value
mysql.default_port => no value => no value
mysql.default_socket => no value => no value
mysql.default_user => no value => no value
mysql.max_links => Unlimited => Unlimited
mysql.max_persistent => Unlimited => Unlimited
mysql.trace_mode => Off => Off

openssl

OpenSSL support => enabled
OpenSSL Version => OpenSSL 0.9.7e 25 Oct 2004

overload

User-Space Object Overloading Support => enabled

pcre

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 4.5 01-December-2003

posix

Revision => $Revision: 1.51.2.2 $

session

Session Support => enabled
Registered save handlers => files user mmcache

Directive => Local Value => Master Value
session.auto_start => Off => Off
session.bug_compat_42 => On => On
session.bug_compat_warn => On => On
session.cache_expire => 180 => 180
session.cache_limiter => nocache => nocache
session.cookie_domain => no value => no value
session.cookie_lifetime => 0 => 0
session.cookie_path => / => /
session.cookie_secure => Off => Off
session.entropy_file => no value => no value
session.entropy_length => 0 => 0
session.gc_divisor => 100 => 100
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.name => PHPSESSID => PHPSESSID
session.referer_check => no value => no value
session.save_handler => files => files
session.save_path => /tmp => /tmp
session.serialize_handler => php => php
session.use_cookies => On => On
session.use_only_cookies => Off => Off
session.use_trans_sid => Off => Off

standard

Regex Library => Bundled library enabled
Dynamic Library Support => enabled
Path to sendmail => /usr/sbin/sendmail -t -i

Directive => Local Value => Master Value
assert.active => 1 => 1
assert.bail => 0 => 0
assert.callback => no value => no value
assert.quiet_eval => 0 => 0
assert.warning => 1 => 1
auto_detect_line_endings => 0 => 0
default_socket_timeout => 60 => 60
safe_mode_allowed_env_vars => PHP_ => PHP_
safe_mode_protected_env_vars => LD_LIBRARY_PATH => LD_LIBRARY_PATH
url_rewriter.tags => a=href,area=href,frame=src,input=src,form=,fieldse t= => a=href,area=href,frame=src,input=src,form=,fieldse t=
user_agent => no value => no value

sysvmsg

sysvmsg support => enabled
Revision => $Revision: 1.4.2.3 $

tokenizer

Tokenizer Support => enabled

xml

XML Support => active
XML Namespace Support => active
EXPAT Version => 1.95.6

zlib

ZLib Support => enabled
Compiled Version => 1.2.2
Linked Version => 1.2.2

Directive => Local Value => Master Value
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Additional Modules

Module Name
sysvsem
sysvshm

Environment

Variable => Value
HZ => 100
SHELL => /bin/bash
TERM => xterm
USER => root
MAIL => /var/mail/root
PATH => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/www/apache/bin:/www/apache/sbin:/www/php/bin
PWD => /root
PS1 => \h:\w\$
SHLVL => 1
HOME => /root
LOGNAME => root
_ => /www/php/bin/php

PHP Variables

Variable => Value
_SERVER["HZ"] => 100
_SERVER["SHELL"] => /bin/bash
_SERVER["TERM"] => xterm
_SERVER["USER"] => root
_SERVER["MAIL"] => /var/mail/root
_SERVER["PATH"] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/www/apache/bin:/www/apache/sbin:/www/php/bin
_SERVER["PWD"] => /root
_SERVER["PS1"] => \\h:\\w\\$
_SERVER["SHLVL"] => 1
_SERVER["HOME"] => /root
_SERVER["LOGNAME"] => root
_SERVER["_"] => /www/php/bin/php
_SERVER["PHP_SELF"] => -
_SERVER["SCRIPT_NAME"] => -
_SERVER["SCRIPT_FILENAME"] =>
_SERVER["PATH_TRANSLATED"] =>
_SERVER["DOCUMENT_ROOT"] =>
_SERVER["argv"] => Array
(
[0] => -
)

_SERVER["argc"] => 1
_ENV["HZ"] => 100
_ENV["SHELL"] => /bin/bash
_ENV["TERM"] => xterm
_ENV["USER"] => root
_ENV["MAIL"] => /var/mail/root
_ENV["PATH"] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/www/apache/bin:/www/apache/sbin:/www/php/bin
_ENV["PWD"] => /root
_ENV["PS1"] => \\h:\\w\\$
_ENV["SHLVL"] => 1
_ENV["HOME"] => /root
_ENV["LOGNAME"] => root
_ENV["_"] => /www/php/bin/php

PHP License
This program is free software; you can redistribute it and/or modify
it under the terms of the PHP License as published by the PHP Group
and included in the distribution in the file: LICENSE

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If you did not receive a copy of the PHP license, or have any
questions about PHP licensing, please contact license@php.net.


11. Apache httpd.conf info:

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
inSpareServers 35
MaxSpareServers 45
StartServers 50
MaxClients 600
MaxRequestsPerChild 25

12. vBulletin 3.0.3 (patched) - I know, we are going to upgrade soon, but this is about performance.

13. Apache log file are rotated (and truncated) daily - no large files.

Additional info:
apache<defunct> - processes indicated apache crashing.
Running 'dmesg' command shows the following:

apache: page allocation failure. order:0, mode:0x20
[<c01426c2>] __alloc_pages+0x1c2/0x370
[<c0142895>] __get_free_pages+0x25/0x40
[<c0145f63>] kmem_getpages+0x23/0xd0
[<c0146c8e>] cache_grow+0xae/0x160
[<c012f96e>] __kernel_text_address+0x2e/0x40
[<c0146ebb>] cache_alloc_refill+0x17b/0x230
[<c01426c2>] __alloc_pages+0x1c2/0x370
[<c0147238>] __kmalloc+0x88/0xa0
[<c0453507>] alloc_skb+0x47/0xf0
[<c0381a87>] e1000_alloc_rx_buffers+0x57/0x100
[<c0381767>] e1000_clean_rx_irq+0x187/0x450
[<c0113321>] smp_apic_timer_interrupt+0xe1/0xf0
[<c014722a>] __kmalloc+0x7a/0xa0
[<c0103a90>] apic_timer_interrupt+0x1c/0x24
[<c038136a>] e1000_intr+0x3a/0x90
[<c013c470>] handle_IRQ_event+0x30/0x70
[<c013c590>] __do_IRQ+0xe0/0x140
[<c0105509>] do_IRQ+0x19/0x30
[<c0103a02>] common_interrupt+0x1a/0x20
[<c0484e00>] tcp_delack_timer+0x40/0x1f0
[<c0484dc0>] tcp_delack_timer+0x0/0x1f0
[<c012734a>] run_timer_softirq+0xda/0x1a0
[<c013c49b>] handle_IRQ_event+0x5b/0x70
[<c0122fca>] __do_softirq+0xba/0xd0
[<c012300d>] do_softirq+0x2d/0x30
[<c010550e>] do_IRQ+0x1e/0x30
[<c0103a02>] common_interrupt+0x1a/0x20
[<c0152b9e>] page_referenced_one+0x6e/0x150
[<c0152ce0>] page_referenced_anon+0x60/0x90
[<c0152e8e>] page_referenced+0xae/0xb0
[<c014a14b>] refill_inactive_zone+0x50b/0x560
[<c014a23b>] shrink_zone+0x9b/0xf0
[<c014a306>] shrink_caches+0x76/0x80
[<c014a3d4>] try_to_free_pages+0xc4/0x1b0
[<c014271b>] __alloc_pages+0x21b/0x370
[<c014dcfb>] do_anonymous_page+0x9b/0x1b0
[<c014de70>] do_no_page+0x60/0x330
[<c014c06f>] pte_alloc_map+0x9f/0xe0
[<c014e346>] handle_mm_fault+0xf6/0x180
[<c0116b2c>] do_page_fault+0x19c/0x5d5
[<c0150f4d>] do_brk+0x1ad/0x280
[<c014f0fe>] sys_brk+0xee/0x120
[<c0116990>] do_page_fault+0x0/0x5d5
[<c0103b37>] error_code+0x2b/0x30

Zachery
Thu 10th Mar '05, 12:45pm
Your swap is not that bad yet.

Turn MaxClients in httpd.conf from 600 to 250 and restart apache

You should upgrade vB to 3.0.7, there are always some preformance enhacnments done between versions.

and otherwise we wait for george.

lewekleonek
Sun 13th Mar '05, 12:37am
bump...
anyone... any more comments - Apache crash - how to resolve it?
eva2000, I count on you.

/LL

Zachery
Sun 13th Mar '05, 2:15pm
Did you try my suggestions?

eva2000
Mon 14th Mar '05, 8:17am
dam

| Max_used_connections | 1024 |

that's 1000+ mysql concurrent connections! that's alot but your problem seems to be web end

i would reduce maxclients from 600 to 200 and restart apache server

on web server install eaccelerator from http://eaccelerator.net/InstallEacceleratorUk read http://www.vbulletin.com/forum/showpost.php?p=805040&postcount=441

with that many users i'd add another 1GB of ram on web server as well

lewekleonek
Mon 14th Mar '05, 1:06pm
Hi,
Yes I've reduced the number of Apache MaxClients from 600 to 200. Took a bit longer before the ringbuffer filled in with apache crash trace calls...

I assume that before I install eAccelerator, I have to remove turck-mmcache.

Thanks again,
/LL

lewekleonek
Mon 14th Mar '05, 1:08pm
BTW: mysql server is doing just fine... i actually upped the max allowed clients from 1024 to 1500... which helped a bit... mysql server is barely utilizing CPU power... and memory usage is not big at all...

/LL

eva2000
Wed 16th Mar '05, 1:37am
max_connections you mean ? leave it at 1024