PDA

View Full Version : Optimization Request


trana
Wed 26th Apr '06, 7:15pm
In order to help you with server problems and optimization, please provide the following info in your thread (courtesy of eva2000):

1. Dedicated server. One other VB site (new test site with no traffic)
2. Athlon XP 2600
1 GB RAM
2x80 GB RAID mirror (in OS, not hardware)
Windows 2003 Server
IIS 6
PHP 5.0.4
MySQL 4.1.11nt

3. No.
4. Windows installer
5. 1200 users, 40-60 online at one time
6.
basedir="C:/Program Files/MySQL/MySQL Server 4.1/"
datadir="C:/Program Files/MySQL/MySQL Server 4.1/Data/"
default-character-set=latin1
default-storage-engine=INNODB
max_connections=100
query_cache_size=0
table_cache=256
tmp_table_size=52M
thread_cache_size=8
myisam_max_sort_file_size=100G
myisam_max_extra_sort_file_size=100G
myisam_sort_buffer_size=103M
key_buffer_size=85M
read_buffer_size=64K
read_rnd_buffer_size=256K
sort_buffer_size=256K
innodb_additional_mem_pool_size=3469K
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=2M
innodb_buffer_pool_size=165M
innodb_log_file_size=83M
innodb_thread_concurrency=8
7.

+---------------------------------+---------------------------------------------
--------------+
| Variable_name | Value
|
+---------------------------------+---------------------------------------------
--------------+
| back_log | 50
|
| basedir | C:\Program Files\MySQL\MySQL Server 4.1\
|
| binlog_cache_size | 32768
|
| bulk_insert_buffer_size | 8388608
|
| character_set_client | latin1
|
| character_set_connection | latin1
|
| character_set_database | latin1
|
| character_set_results | latin1
|
| character_set_server | latin1
|
| character_set_system | utf8
|
| character_sets_dir | C:\Program Files\MySQL\MySQL Server 4.1\shar
e\charsets/ |
| collation_connection | latin1_swedish_ci
|
| collation_database | latin1_swedish_ci
|
| collation_server | latin1_swedish_ci
|
| concurrent_insert | ON
|
| connect_timeout | 5
|
| datadir | C:\Program Files\MySQL\MySQL Server 4.1\Data
\ |
| date_format | %Y-%m-%d
|
| datetime_format | %Y-%m-%d %H:%i:%s
|
| default_week_format | 0
|
| delay_key_write | ON
|
| delayed_insert_limit | 100
|
| delayed_insert_timeout | 300
|
| delayed_queue_size | 1000
|
| expire_logs_days | 0
|
| flush | OFF
|
| flush_time | 1800
|
| ft_boolean_syntax | + -><()~*:""&|
|
| ft_max_word_len | 84
|
| ft_min_word_len | 4
|
| ft_query_expansion_limit | 20
|
| ft_stopword_file | (built-in)
|
| group_concat_max_len | 1024
|
| have_archive | NO
|
| have_bdb | NO
|
| have_blackhole_engine | NO
|
| have_compress | YES
|
| have_crypt | NO
|
| have_csv | NO
|
| have_example_engine | NO
|
| have_geometry | YES
|
| have_innodb | YES
|
| have_isam | NO
|
| have_ndbcluster | NO
|
| have_openssl | NO
|
| have_query_cache | YES
|
| have_raid | NO
|
| have_rtree_keys | YES
|
| have_symlink | YES
|
| init_connect |
|
| init_file |
|
| init_slave |
|
| innodb_additional_mem_pool_size | 3552256
|
| innodb_autoextend_increment | 8
|
| innodb_buffer_pool_awe_mem_mb | 0
|
| innodb_buffer_pool_size | 173015040
|
| innodb_data_file_path | ibdata1:10M:autoextend
|
| innodb_data_home_dir |
|
| innodb_fast_shutdown | ON
|
| innodb_file_io_threads | 4
|
| innodb_file_per_table | OFF
|
| innodb_flush_log_at_trx_commit | 1
|
| innodb_flush_method |
|
| innodb_force_recovery | 0
|
| innodb_lock_wait_timeout | 50
|
| innodb_locks_unsafe_for_binlog | OFF
|
| innodb_log_arch_dir |
|
| innodb_log_archive | OFF
|
| innodb_log_buffer_size | 2097152
|
| innodb_log_file_size | 87031808
|
| innodb_log_files_in_group | 2
|
| innodb_log_group_home_dir | .\
|
| innodb_max_dirty_pages_pct | 90
|
| innodb_max_purge_lag | 0
|
| innodb_mirrored_log_groups | 1
|
| innodb_open_files | 300
|
| innodb_table_locks | ON
|
| innodb_thread_concurrency | 8
|
| interactive_timeout | 28800
|
| join_buffer_size | 131072
|
| key_buffer_size | 89128960
|
| key_cache_age_threshold | 300
|
| key_cache_block_size | 1024
|
| key_cache_division_limit | 100
|
| language | C:\Program Files\MySQL\MySQL Server 4.1\shar
e\english\ |
| large_files_support | ON
|
| license | GPL
|
| local_infile | ON
|
| log | OFF
|
| log_bin | OFF
|
| log_error | .\acropora.err
|
| log_slave_updates | OFF
|
| log_slow_queries | OFF
|
| log_update | OFF
|
| log_warnings | 1
|
| long_query_time | 10
|
| low_priority_updates | OFF
|
| lower_case_file_system | OFF
|
| lower_case_table_names | 1
|
| max_allowed_packet | 1048576
|
| max_binlog_cache_size | 4294967295
|
| max_binlog_size | 1073741824
|
| max_connect_errors | 10
|
| max_connections | 100
|
| max_delayed_threads | 20
|
| max_error_count | 64
|
| max_heap_table_size | 16777216
|
| max_insert_delayed_threads | 20
|
| max_join_size | 4294967295
|
| max_length_for_sort_data | 1024
|
| max_relay_log_size | 0
|
| max_seeks_for_key | 4294967295
|
| max_sort_length | 1024
|
| max_tmp_tables | 32
|
| max_user_connections | 0
|
| max_write_lock_count | 4294967295
|
| myisam_data_pointer_size | 4
|
| myisam_max_extra_sort_file_size | 107374182400
|
| myisam_max_sort_file_size | 107374182400
|
| myisam_recover_options | OFF
|
| myisam_repair_threads | 1
|
| myisam_sort_buffer_size | 108003328
|
| named_pipe | OFF
|
| net_buffer_length | 16384
|
| net_read_timeout | 30
|
| net_retry_count | 10
|
| net_write_timeout | 60
|
| new | OFF
|
| old_passwords | OFF
|
| open_files_limit | 622
|
| pid_file | C:\Program Files\MySQL\MySQL Server 4.1\Data
\acropora.pid |
| port | 3306
|
| preload_buffer_size | 32768
|
| protocol_version | 10
|
| query_alloc_block_size | 8192
|
| query_cache_limit | 1048576
|
| query_cache_min_res_unit | 4096
|
| query_cache_size | 0
|
| query_cache_type | ON
|
| query_cache_wlock_invalidate | OFF
|
| query_prealloc_size | 8192
|
| range_alloc_block_size | 2048
|
| read_buffer_size | 61440
|
| read_only | OFF
|
| read_rnd_buffer_size | 258048
|
| relay_log_purge | ON
|
| relay_log_space_limit | 0
|
| rpl_recovery_rank | 0
|
| secure_auth | OFF
|
| shared_memory | OFF
|
| shared_memory_base_name | MYSQL
|
| server_id | 0
|
| skip_external_locking | ON
|
| skip_networking | OFF
|
| skip_show_database | OFF
|
| slave_net_timeout | 3600
|
| slave_transaction_retries | 0
|
| slow_launch_time | 2
|
| sort_buffer_size | 262136
|
| sql_mode |
|
| storage_engine | InnoDB
|
| sql_notes | ON
|
| sql_warnings | ON
|
| sync_binlog | 0
|
| sync_replication | 0
|
| sync_replication_slave_id | 0
|
| sync_replication_timeout | 0
|
| sync_frm | ON
|
| system_time_zone | Eastern Standard Time
|
| table_cache | 256
|
| table_type | InnoDB
|
| thread_cache_size | 8
|
| thread_stack | 196608
|
| time_format | %H:%i:%s
|
| time_zone | SYSTEM
|
| tmp_table_size | 54525952
|
| tmpdir |
|
| transaction_alloc_block_size | 8192
|
| transaction_prealloc_size | 4096
|
| tx_isolation | REPEATABLE-READ
|
| version | 4.1.11-nt
|
| version_comment | Official MySQL binary
|
| version_compile_machine | ia32
|
| version_compile_os | Win32
|
| wait_timeout | 28800
|
+---------------------------------+---------------------------------------------
--------------+
+----------------------------+------------+
| Variable_name | Value |
+----------------------------+------------+
| Aborted_clients | 464 |
| Aborted_connects | 1 |
| Binlog_cache_disk_use | 0 |
| Binlog_cache_use | 0 |
| Bytes_received | 2923036169 |
| Bytes_sent | 2302626478 |
| Com_admin_commands | 0 |
| Com_alter_db | 0 |
| Com_alter_table | 355 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 51 |
| Com_change_db | 7782072 |
| Com_change_master | 0 |
| Com_check | 1 |
| Com_checksum | 0 |
| Com_commit | 51 |
| Com_create_db | 1 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 185 |
| Com_dealloc_sql | 0 |
| Com_delete | 368509 |
| Com_delete_multi | 0 |
| Com_do | 0 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 173 |
| Com_drop_user | 0 |
| Com_execute_sql | 0 |
| Com_flush | 0 |
| Com_grant | 0 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_help | 0 |
| Com_insert | 1298560 |
| Com_insert_select | 6519 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 372 |
| Com_optimize | 9 |
| Com_preload_keys | 0 |
| Com_prepare_sql | 0 |
| Com_purge | 0 |
| Com_purge_before_date | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 550695 |
| Com_replace_select | 0 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 74406598 |
| Com_set_option | 18144 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 19 |
| Com_show_charsets | 287 |
| Com_show_collations | 287 |
| Com_show_column_types | 0 |
| Com_show_create_db | 19 |
| Com_show_create_table | 16820 |
| Com_show_databases | 117 |
| Com_show_errors | 0 |
| Com_show_fields | 16879 |
| Com_show_grants | 19 |
| Com_show_innodb_status | 0 |
| Com_show_keys | 138 |
| Com_show_logs | 0 |
| Com_show_master_status | 0 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_privileges | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 5 |
| Com_show_storage_engines | 2 |
| Com_show_tables | 18172 |
| Com_show_variables | 844 |
| Com_show_warnings | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 10 |
| Com_unlock_tables | 372 |
| Com_update | 14938317 |
| Com_update_multi | 0 |
| Connections | 8603800 |
| Created_tmp_disk_tables | 531886 |
| Created_tmp_files | 123027 |
| Created_tmp_tables | 1298148 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Flush_commands | 1 |
| Handler_commit | 197 |
| Handler_delete | 2001054 |
| Handler_discover | 0 |
| Handler_read_first | 21391851 |
| Handler_read_key | 858263453 |
| Handler_read_next | 900039457 |
| Handler_read_prev | 108638134 |
| Handler_read_rnd | 51222615 |
| Handler_read_rnd_next | 4193326488 |
| Handler_rollback | 695858 |
| Handler_update | 186520866 |
| Handler_write | 792660630 |
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 74573 |
| Key_blocks_used | 26297 |
| Key_read_requests | 1720123928 |
| Key_reads | 5455753 |
| Key_write_requests | 7890726 |
| Key_writes | 3449439 |
| Max_used_connections | 36 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 339 |
| Open_streams | 0 |
| Open_tables | 197 |
| Opened_tables | 291117 |
| Qcache_free_blocks | 0 |
| Qcache_free_memory | 0 |
| Qcache_hits | 0 |
| Qcache_inserts | 0 |
| Qcache_lowmem_prunes | 0 |
| Qcache_not_cached | 0 |
| Qcache_queries_in_cache | 0 |
| Qcache_total_blocks | 0 |
| Questions | 108033133 |
| Rpl_status | NULL |
| Select_full_join | 42463 |
| Select_full_range_join | 1 |
| Select_range | 17874672 |
| Select_range_check | 0 |
| Select_scan | 13610733 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slave_retried_transactions | 0 |
| Slow_launch_threads | 32 |
| Slow_queries | 305 |
| Sort_merge_passes | 88559 |
| Sort_range | 3034788 |
| Sort_rows | 1419194959 |
| Sort_scan | 6935909 |
| Table_locks_immediate | 128077042 |
| Table_locks_waited | 1336 |
| Threads_cached | 6 |
| Threads_connected | 2 |
| Threads_created | 510 |
| Threads_running | 2 |
| Uptime | 5009260 |
+----------------------------+------------+
Uptime: 5009261 Threads: 1 Questions: 108033153 Slow queries: 305 Opens: 291
117 Flush tables: 1 Open tables: 197 Queries per second avg: 21.567
mysqladmin Ver 8.41 Distrib 4.1.11, for Win32 on ia32
Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license
Server version 4.1.11-nt
Protocol version 10
Connection localhost via TCP/IP
TCP port 3306
Uptime: 57 days 23 hours 27 min 41 sec
Threads: 1 Questions: 108033153 Slow queries: 305 Opens: 291117 Flush tables
: 1 Open tables: 197 Queries per second avg: 21.567
C:\Documents and Settings\Administrator>
Quote:
mysqladmin -u root -p var ext stat ver

8. Coppermine is installed but only used by 2 members and infrequently.

9. average 40-60 max 100+

11. IIS/NT equivalent?
12. 3.5.1
13. None.
14. IIS/NT equivalent?

Anything I missed? PM me.

Thanks in advance!

eva2000
Thu 27th Apr '06, 11:24am
Any reason you're using default innodb tables and no the usual MyISAM ?

when you look at your data files in C:/Program Files/MySQL/MySQL Server 4.1/Data what are the extensions of the mysql database files ? *.myd, *.myi and *.frm ?

1. Upgrade MySQL server to 4.1.18 http://www.vbulletin.com/forum/showthread.php?t=175163
2. Upgrade PHP to 5.1.2 at least
3. Edit your my.ini and place the following mysql server settings in your my.ini and restart mysql server afterwards


[mysqld]
basedir="C:/Program Files/MySQL/MySQL Server 4.1/"
datadir="C:/Program Files/MySQL/MySQL Server 4.1/Data/"
safe-show-database
back_log = 75
skip-innodb
max_connections = 400
key_buffer = 32M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1800
thread_cache_size = 256
wait_timeout = 60
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
max_connect_errors = 10
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 2M
query_cache_size = 64M
query_cache_type = 1
query_prealloc_size = 163840
query_alloc_block_size = 32768
default-storage-engine = MyISAM

[mysqld_safe]
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M

[mysqlhotcopy]
interactive-timeout


4. Upgrade to vB vB 3.5.4 http://www.vbulletin.com/forum/showthread.php?t=176170 if you're on vB 3.5.x

5. If you just upgraded to vB 3.5.x try to disable these 4 options:

Admin CP -> vBulletin Options -> Forums Home Page Options -> Display Logged in Users?

Admin CP -> vBulletin Options -> Forum Display Options (forumdisplay) -> Show Users Browsing Forums

Admin CP -> vBulletin Options -> Thread Display Options -> Show Users Browsing Thread

Admin CP -> vBulletin Options -> Message Searching Options -> Automatic Similar Thread search

trana
Sat 29th Apr '06, 1:05pm
2. Upgrade PHP to 5.1.2 at least


What is the recommended PHP 5 release for VB?

Steve Machol
Sat 29th Apr '06, 1:22pm
What is the recommended PHP 5 release for VB?
Sorry, what do you mean? :confused:

trana
Sat 29th Apr '06, 1:23pm
Well, he said "5.1.2 at least". Is the recommended level 5.1.2 or something higher?

Steve Machol
Sat 29th Apr '06, 1:28pm
Generally yes, although the latest versions can sometimes introduce new bugs and incompatibilities. I usually stick with the newest stable and proven version unless there is a specific security issue.

eva2000
Sun 30th Apr '06, 1:15am
What is the recommended PHP 5 release for VB?
well for windows usage 5.1.2 would be best

trana
Thu 11th May '06, 12:55am
OK, I completed all of your suggestions except upgrading to 3.5.4. I am still running 3.5.1 for the time being until I get a couple of add ons worked out.

Your suggestions were primarily for optimizing MySQL. I took a look at my proc utilization and see that w3wp.exe is killing it, routinely taking 50-100% of processor time.

I disabled a hack that I thought might be causing this (vbshout) and it had no impact.

I was wondering if someone can tell me how to really debug the PHP/IIS process to see what exactly is taking so much time. I don't think that this is normal as I am only running with 40-80 users online at once.

eva2000
Thu 11th May '06, 8:26am
what is exactly the problem ? what is taking so much time ?

trana
Thu 11th May '06, 11:57am
Thats my question.

The w3wp.exe process sits between 50-100%. This is the IIS/ISAPI process and all subthreads. I was wondering if someone can advise on how to identify the particular ISAPI directives or PHP calls that are taking the most amount of time.

eva2000
Fri 12th May '06, 7:12am
Do you have hacks or other non-vB php scripts/code running on server ?

no idea but try google http://www.google.com/search?q=w3wp.exe+100%25+cpu+usage&btnG=Search&hl=en&lr=lang_en and win2003 mailing lists searches for w3wp.exe

i.e.

http://www.coveo.com/forums/ShowPost.aspx?PostID=723

http://www.velocityreviews.com/forums/t109263-w3wpexe-100-cpu.html

If it was me, i'd just dump IIS6 since i don't know much about it and use Apache hehe