PDA

View Full Version : Optimization request please - thx


jwocky
Tue 18th Apr '06, 3:59pm
Ok here goes:

1) Dedicated Server
2)

Dual AMD Opteron 244
4 gigs DDR RAM
200 Gigs SATA RAID-1

FreeBSD 5.4 (32bit)
Apache v 1.3.34
PHP 4.4.2
MySQL 4.1.14

3) No InnoDB
4) Not sure
5)

last pid: 9474; load averages: 5.40, 3.99, 3.40 up 0+17:37:16 14:43:54
258 processes: 14 running, 244 sleeping
CPU states: 86.1% user, 0.0% nice, 11.6% system, 2.3% interrupt, 0.0% idle
Mem: 567M Active, 2875M Inact, 202M Wired, 148M Cache, 112M Buf, 129M Free
Swap: 2048M Total, 8K Used, 2048M Free
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
9437 apache 4 0 24552K 18332K sbwait 0:01 1.78% 1.51% httpd
9403 apache 99 0 23728K 18460K RUN 0:01 1.65% 1.42% httpd
9460 apache 100 0 23960K 18688K RUN 0:01 1.81% 1.32% httpd
9374 apache 4 0 23532K 18228K sbwait 0:01 1.20% 1.07% httpd
9467 apache 99 0 21632K 16324K RUN 0:00 1.48% 1.07% httpd
9471 apache 100 0 23764K 18400K RUN 0:00 1.48% 1.07% httpd
9419 apache 4 0 24580K 18352K sbwait 0:01 1.21% 1.03% httpd
9410 apache 4 0 23924K 18652K sbwait 0:01 1.21% 1.03% httpd


6)

[mysqld]
safe-show-database
skip-innodb
max_connections = 650
key_buffer = 64M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 384
wait_timeout = 90
connect_timeout = 10
tmp_table_size = 128M
max_heap_table_size = 128M
max_allowed_packet = 64M
max_connect_errors = 10
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 8M
query_cache_size = 80M
query_cache_type = 1
query_prealloc_size = 163840
query_alloc_block_size = 32768
[mysqld_safe]
nice = -5
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M
[mysqlhotcopy]
interactive-timeout



+---------------------------------+------------------------------------------------------------
| Variable_name | Value
+---------------------------------+------------------------------------------------------------
| back_log | 50
| basedir | /usr/local/mysql-standard-4.1.14-unknown-freebsd5.3-i386/
| binlog_cache_size | 32768
| bulk_insert_buffer_size | 8388608
| character_set_client | latin1
| character_set_connection | latin1
| character_set_database | latin1
| character_set_results | latin1
| character_set_server | latin1
| character_set_system | utf8
| character_sets_dir | /usr/local/mysql-standard-4.1.14-unknown-freebsd5.3-i386/sh
| collation_connection | latin1_swedish_ci
| collation_database | latin1_swedish_ci
| collation_server | latin1_swedish_ci
| concurrent_insert | ON
| connect_timeout | 10
| datadir | /usr/local/mysql/data/
| date_format | %Y-%m-%d
| datetime_format | %Y-%m-%d %H:%i:%s
| default_week_format | 0
| delay_key_write | ON
| delayed_insert_limit | 100
| delayed_insert_timeout | 300
| delayed_queue_size | 1000
| expire_logs_days | 0
| flush | OFF
| flush_time | 0
| ft_boolean_syntax | + -><()~*:""&|
| ft_max_word_len | 84
| ft_min_word_len | 4
| ft_query_expansion_limit | 20
| ft_stopword_file | (built-in)
| group_concat_max_len | 1024
| have_archive | YES
| have_bdb | NO
| have_blackhole_engine | NO
| have_compress | YES
| have_crypt | YES
| have_csv | NO
| have_example_engine | NO
| have_geometry | YES
| have_innodb | DISABLED
| have_isam | NO
| have_ndbcluster | NO
| have_openssl | NO
| have_query_cache | YES
| have_raid | YES
| have_rtree_keys | YES
| have_symlink | YES
| init_connect |
| init_file |
| init_slave |
| innodb_additional_mem_pool_size | 1048576
| innodb_autoextend_increment | 8
| innodb_buffer_pool_awe_mem_mb | 0
| innodb_buffer_pool_size | 8388608
| innodb_data_file_path |
| innodb_data_home_dir |
| innodb_fast_shutdown | ON
| innodb_file_io_threads | 4
| innodb_file_per_table | OFF
| innodb_flush_log_at_trx_commit | 1
| innodb_flush_method |
| innodb_force_recovery | 0
| innodb_lock_wait_timeout | 50
| innodb_locks_unsafe_for_binlog | OFF
| innodb_log_arch_dir |
| innodb_log_archive | OFF
| innodb_log_buffer_size | 1048576
| innodb_log_file_size | 5242880
| innodb_log_files_in_group | 2
| innodb_log_group_home_dir |
| innodb_max_dirty_pages_pct | 90
| innodb_max_purge_lag | 0
| innodb_mirrored_log_groups | 1
| innodb_open_files | 300
| innodb_table_locks | ON
| innodb_thread_concurrency | 8
| interactive_timeout | 28800
| join_buffer_size | 1044480
| key_buffer_size | 67108864
| key_cache_age_threshold | 300
| key_cache_block_size | 1024
| key_cache_division_limit | 100
| language | /usr/local/mysql-standard-4.1.14-unknown-freebsd5.3-i386/sh
| large_files_support | ON
| license | GPL
| local_infile | ON
| locked_in_memory | OFF
| log | OFF
| log_bin | OFF
| log_error |
| log_slave_updates | OFF
| log_slow_queries | OFF
| log_update | OFF
| log_warnings | 1
| long_query_time | 10
| low_priority_updates | OFF
| lower_case_file_system | OFF
| lower_case_table_names | 0
| max_allowed_packet | 67107840
| max_binlog_cache_size | 4294967295
| max_binlog_size | 1073741824
| max_connect_errors | 10
| max_connections | 650
| max_delayed_threads | 20
| max_error_count | 64
| max_heap_table_size | 134216704
| max_insert_delayed_threads | 20
| max_join_size | 4294967295
| max_length_for_sort_data | 1024
| max_relay_log_size | 0
| max_seeks_for_key | 4294967295
| max_sort_length | 1024
| max_tmp_tables | 32
| max_user_connections | 0
| max_write_lock_count | 4294967295
| myisam_data_pointer_size | 4
| myisam_max_extra_sort_file_size | 2147483648
| myisam_max_sort_file_size | 2147483647
| myisam_recover_options | OFF
| myisam_repair_threads | 1
| myisam_sort_buffer_size | 67108864
| net_buffer_length | 16384
| net_read_timeout | 30
| net_retry_count | 1000000
| net_write_timeout | 60
| new | OFF
| old_passwords | OFF
| open_files_limit | 11095
| port | 3306
| preload_buffer_size | 32768
| protocol_version | 10
| query_alloc_block_size | 32768
| query_cache_limit | 8388608
| query_cache_min_res_unit | 4096
| query_cache_size | 83886080
| query_cache_type | ON
| query_cache_wlock_invalidate | OFF
| query_prealloc_size | 163840
| range_alloc_block_size | 2048
| read_buffer_size | 1044480
| read_only | OFF
| read_rnd_buffer_size | 520192
| relay_log_purge | ON
| relay_log_space_limit | 0
| rpl_recovery_rank | 0
| secure_auth | OFF
| server_id | 0
| skip_external_locking | ON
| skip_networking | OFF
| skip_show_database | OFF
| slave_net_timeout | 3600
| slave_transaction_retries | 0
| slow_launch_time | 2
| socket | /tmp/mysql.sock
| sort_buffer_size | 2097144
| sql_mode |
| storage_engine | MyISAM
| sql_notes | ON
| sql_warnings | ON
| sync_binlog | 0
| sync_replication | 0
| sync_replication_slave_id | 0
| sync_replication_timeout | 0
| sync_frm | ON
| system_time_zone | EDT
| table_cache | 1800
| table_type | MyISAM
| thread_cache_size | 384
| thread_stack | 196608
| time_format | %H:%i:%s
| time_zone | SYSTEM
| tmp_table_size | 134217728
| tmpdir |
| transaction_alloc_block_size | 8192
| transaction_prealloc_size | 4096
| tx_isolation | REPEATABLE-READ
| version | 4.1.14-standard
| version_comment | MySQL Community Edition - Standard (GPL)
| version_compile_machine | i386
| version_compile_os | unknown-freebsd5.3
| wait_timeout | 90
+---------------------------------+------------------------------------------------------------
+----------------------------+-----------+
| Variable_name | Value |
+----------------------------+-----------+
| Aborted_clients | 1399 |
| Aborted_connects | 3 |
| Binlog_cache_disk_use | 0 |
| Binlog_cache_use | 0 |
| Bytes_received | 493752721 |
| Bytes_sent | 947155676 |
| Com_admin_commands | 0 |
| Com_alter_db | 0 |
| Com_alter_table | 202 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 158102 |
| Com_change_master | 0 |
| Com_check | 1 |
| Com_checksum | 0 |
| Com_commit | 0 |
| Com_create_db | 2 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 101 |
| Com_dealloc_sql | 0 |
| Com_delete | 576 |
| Com_delete_multi | 0 |
| Com_do | 0 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 101 |
| Com_drop_user | 0 |
| Com_execute_sql | 0 |
| Com_flush | 2 |
| Com_grant | 4 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_help | 0 |
| Com_insert | 60043 |
| Com_insert_select | 58 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 101 |
| Com_optimize | 0 |
| Com_preload_keys | 0 |
| Com_prepare_sql | 0 |
| Com_purge | 0 |
| Com_purge_before_date | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 1193 |
| Com_replace_select | 0 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 191682 |
| Com_set_option | 70 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 3 |
| Com_show_charsets | 10 |
| Com_show_collations | 10 |
| Com_show_column_types | 0 |
| Com_show_create_db | 0 |
| Com_show_create_table | 0 |
| Com_show_databases | 19 |
| Com_show_errors | 0 |
| Com_show_fields | 0 |
| Com_show_grants | 4 |
| Com_show_innodb_status | 0 |
| Com_show_keys | 87 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_privileges | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 7 |
| Com_show_storage_engines | 0 |
| Com_show_tables | 23 |
| Com_show_variables | 34 |
| Com_show_warnings | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_stmt_prepare | 0 |
| Com_stmt_execute | 0 |
| Com_stmt_send_long_data | 0 |
| Com_stmt_reset | 0 |
| Com_stmt_close | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 101 |
| Com_update | 96504 |
| Com_update_multi | 0 |
| Connections | 158352 |
| Created_tmp_disk_tables | 100 |
| Created_tmp_files | 198 |
| Created_tmp_tables | 4999 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Flush_commands | 1 |
| Handler_commit | 0 |
| Handler_delete | 8604 |
| Handler_discover | 0 |
| Handler_read_first | 31452 |
| Handler_read_key | 20839077 |
| Handler_read_next | 19697921 |
| Handler_read_prev | 81099 |
| Handler_read_rnd | 1528052 |
| Handler_read_rnd_next | 19481111 |
| Handler_rollback | 0 |
| Handler_update | 120082 |
| Handler_write | 6481582 |
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 0 |
| Key_blocks_used | 57990 |
| Key_read_requests | 65390467 |
| Key_reads | 167950 |
| Key_write_requests | 6350788 |
| Key_writes | 143462 |
| Max_used_connections | 110 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 324 |
| Open_streams | 0 |
| Open_tables | 254 |
| Opened_tables | 578 |
| Qcache_free_blocks | 1279 |
| Qcache_free_memory | 69547144 |
| Qcache_hits | 656246 |
| Qcache_inserts | 185831 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 5851 |
| Qcache_queries_in_cache | 3357 |
| Qcache_total_blocks | 8338 |
| Questions | 1322898 |
| Rpl_status | NULL |
| Select_full_join | 5 |
| Select_full_range_join | 0 |
| Select_range | 29512 |
| Select_range_check | 0 |
| Select_scan | 13377 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slave_retried_transactions | 0 |
| Slow_launch_threads | 0 |
| Slow_queries | 0 |
| Sort_merge_passes | 99 |
| Sort_range | 32580 |
| Sort_rows | 5121176 |
| Sort_scan | 5883 |
| Table_locks_immediate | 743076 |
| Table_locks_waited | 1945 |
| Threads_cached | 87 |
| Threads_connected | 23 |
| Threads_created | 110 |
| Threads_running | 4 |
| Uptime | 60491 |
+----------------------------+-----------+
Uptime: 60491 Threads: 23 Questions: 1322899 Slow queries: 0 Opens: 578 Flush tables: 1 O
mysqladmin Ver 8.41 Distrib 4.1.14, for unknown-freebsd5.3 on i386
Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license





8. Yes just vB, some little things but are used very very very infrequently

9. Cookie timeout = 1100. Avg users is about 600 users total.

11)



Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 400
MaxRequestsPerChild 0

12) vb 3.0.7 (yea I know upgrade will be coming soon :)

13) Nope none found

14)

FreeBSD 343151.ds.nac.net 5.4-RELEASE FreeBSD 5.4-RELEASE #0: Sun May 8 10:21:06 UTC 2005 root@harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386

** ulimit and cat commands didnt work on my server :(

If they deliniate the CPU this computer is running on a Dual Operon 244 CPU as per eva's high recommendations.

Thanks for anyhelp you can give, the server has been running quite slow :(

eva2000
Wed 19th Apr '06, 9:57am
Those cpu loads look way too high.. how's FreeBSD's support for dual opteron cpus ? Are you sure both cpus are recognised or utilised there ? Could be OS related ?

1. Upgrade MySQL server to 4.1.18 http://www.vbulletin.com/forum/showthread.php?t=175163
2. Edit /etc/my.cnf and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards


[mysqld]
safe-show-database
skip-innodb
max_connections = 650
key_buffer = 192M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 384
wait_timeout = 60
connect_timeout = 10
tmp_table_size = 256M
max_heap_table_size = 256M
max_allowed_packet = 64M
max_connect_errors = 10
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 8M
query_cache_size = 160M
query_cache_type = 1
query_prealloc_size = 163840
query_alloc_block_size = 32768

[mysqld_safe]
nice = -10
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 384M
sort_buffer = 384M
read_buffer = 256M
write_buffer = 256M

[mysqlhotcopy]
interactive-timeout


3. Install APC Cache lastest version from http://pecl.php.net/package/APC. Read install guide at http://www.vbulletin.com/forum/showthread.php?t=165367 - please remove Zend Optimizer from php.ini before installing APC Cache

4. Upgrade to vB 3.0.13 http://www.vbulletin.com/forum/showthread.php?t=176176

5. Edit httpd.conf values from

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 400
MaxRequestsPerChild 0

to

Timeout 300
KeepAlive On
MaxKeepAliveRequests 120
KeepAliveTimeout 5
MinSpareServers 15
MaxSpareServers 20
StartServers 15
MaxClients 180
MaxRequestsPerChild 10000

restart apache

jwocky
Wed 19th Apr '06, 4:11pm
Eva thanks so much for the quick respons, you really are a godsend to this community.

Some followups...

0) Yea im not realy sure how FreeBSD handles the dual CPU setup, wish I knew how to track this down but i'm quite a novice at this sometimes. Im at this point *hoping* that the OS is utilizing my resources properly :)

1) Upgrade went smoothly, thanks for the advice!

2) This made a BIG difference. The site is not only very usable now, its actually quite fast now (with about 600-700 users)... thanks so much, i knew my settings were all haywire. The server loads now sit at around 3 and spike up to 6 every so often, but generally about 3-4.

3) This is my next step, that guide is nice but unfortunatly I cant get past the phpize step because something about "Cannot find autoconfig". and $PHP_AUTOCONF

5) Done. Awesome!!

Overall, thanks for all the help, just wanted to give you the followup that the server is running alot faster now and that server loads are now down to the 3-4 range. The help is so appreciated.!

eva2000
Thu 20th Apr '06, 9:25am
awesome stuff.. yes my.cnf settings are very important to have set correctly :)

maybe google or ask web host or post over at webhostingtalk.com on how freebsd and dual opteron cpus are handled :)