vb 2.2.4 / RaQ4r / huge backend

[mixylplik3]

I recently picked up a new hosting setup for various clients. The setup is: RaQ4r (front-end, Apache 1.3.20 and PHP 4.1.2 with 512MB of RAM) and a rather sizeable multi-processor Athlon machine with the latest MySQL (3.23.47). I moved from a single RaQ4i that had the db and everything on it.

My new setup is slower than sin, and my CPU bogs like crazy. This should definitely not be the case. The RaQ is connected to the backend on a full-duplex 100base-T connection. HTTPD processes take anywhere from 60% to 95% of CPU time and the RaQ is constantly getting slammed.

Things I've tried: installed Zend Optimizer, mod_gzip, played with httpd.conf settings (but found the default ones were fastest), and played with the my.cnf file a bit but to no avail. I also cleaned out the thread subscriptions, but my mail server has been fine. This is very frustrating, because vb is working fine.

Is vb not too keen on connecting to a LAN-based box? I'm out of ideas and would appreciate any feedback. I have tried searching through this forum and couldn't find anything too helpful.

Thanks everyone...

Update: Another thing is that this is intermittent, but is an issue more often than not. It basically seems to have to do with HTTPD processes not dying, and sticking around for hours sucking up memory and CPU time. Persistant connections are on as well.

[eva2000]

let me get this straight you have 2 servers one web and one for database right ?

can you provide specs for both and for your database server please provide the following

1. your server specs, such as mysql and php version
2. if possible how mysql was compiled/installed
3. your top stats
4. your mysql configuration variables located at /etc/my.cnf or c:\my.cnf if on Windows server if you don't have that file you need to log into telnet and as root user type

mysqladmin -u root -p variables

copy and paste output here

5. your mysql extended-status output either still telnet as root user type

mysqladmin -u root -p extended-status

copy and paste output here

or preferred is to installed extended-status output script which is located at http://vbulletin.com/forum/showthread.php?threadid=3477

and post url to that here

6. oh and is your vB the only thing on the server? or other scripts? sites?

7. how many average and max concurrent users on your vB forum ?

8. create a file named phpinfo.php and place this code in it and post the url/link to it from your web site

<?
phpinfo();
?>

i.e. yourdomain.com/phpinfo.php

[mixylplik3]

1. your server specs, such as mysql and php version

RaQ 4r (front end) 450mhz K6-3, 512MB RAM, dual 30GB HD, PHP 4.1.2 (http://ecin.net/phpinfo.php)
DB machine - dual Athlon MP 2000, 1024MB RAM, MySQL 3.23.47, Apache 1.3.23 (http://peko.ecin.net/info.php)

if possible how mysql was compiled/installed

Not totally sure. Can find out.

your top stats

Partial top stats for this moment:
11:43pm up 5 days, 21:30, 2 users, load average: 18.04, 12.79, 11.03
195 processes: 186 sleeping, 9 running, 0 zombie, 0 stopped
CPU states: 65.3% user, 25.0% system, 0.0% nice, 9.6% idle
Mem: 517112K av, 440476K used, 76636K free, 423048K shrd, 57064K buff
Swap: 131448K av, 26028K used, 105420K free 145188K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
23843 httpd 7 0 11640 8400 4988 R 0 14.2 1.6 4:46 httpd
14468 httpd 10 0 9796 4884 3124 R 0 6.7 0.9 0:00 httpd
14398 admin 10 0 932 932 676 R 0 4.8 0.1 0:01 top
14456 httpd 1 0 8908 4144 3352 S 0 3.6 0.8 0:00 httpd
14474 httpd 5 0 8740 3844 3252 S 0 2.8 0.7 0:00 httpd
14475 httpd 5 0 8740 3844 3252 S 0 2.8 0.7 0:00 httpd
14482 httpd 9 0 8728 3832 3240 S 0 2.8 0.7 0:00 httpd
14469 httpd 4 0 8740 3868 3272 S 0 2.6 0.7 0:00 httpd
14485 httpd 9 0 8740 3844 3252 S 0 2.6 0.7 0:00 httpd
2630 httpd 0 0 11420 8164 5020 S 0 2.4 1.5 2:30 httpd
14489 httpd 13 0 9124 4412 3292 R 0 2.4 0.8 0:00 httpd
14296 httpd 1 0 9960 5140 3372 S 0 2.3 0.9 0:03 httpd
14467 httpd 3 0 8740 3868 3272 S 0 2.3 0.7 0:00 httpd
14473 httpd 4 0 8740 3844 3252 S 0 2.3 0.7 0:00 httpd
14479 httpd 8 0 8728 3832 3240 S 0 2.3 0.7 0:00 httpd
14500 root 14 0 420 420 348 R 0 2.3 0.0 0:00 pidof
14465 httpd 2 0 8740 3844 3252 S 0 2.1 0.7 0:00 httpd
14478 httpd 7 0 8728 3832 3240 S 0 2.1 0.7 0:00 httpd
14484 httpd 8 0 8728 3832 3240 S 0 1.8 0.7 0:00 httpd
14491 httpd 14 0 8836 4228 3444 R 0 1.8 0.8 0:00 httpd
23847 httpd 5 0 11468 8384 5144 S 0 1.4 1.6 4:59 httpd
14493 httpd 14 0 8808 3848 3076 R 0 1.4 0.7 0:00 httpd
14492 httpd 14 0 8824 3852 3056 R 0 1.3 0.7 0:00 httpd
14477 httpd 5 0 7784 2604 2336 S 0 1.1 0.5 0:00 httpd
14634 httpd 1 0 10460 7056 3668 S 0 0.8 1.3 13:45 httpd
785 root 10 0 6408 1072 1064 R 0 0.6 0.2 1:18 httpd
14481 httpd 6 0 7780 2824 2556 S 0 0.6 0.5 0:00 httpd
3237 httpd 0 0 11624 8512 5116 S 0 0.4 1.6 2:35 httpd
14486 root 7 0 820 820 752 S 0 0.4 0.1 0:00 lcdstop
30889 httpd 1 0 11508 8424 5212 S 0 0.3 1.6 3:01 httpd
14241 httpd 1 0 11008 7456 4564 S 0 0.3 1.4 0:02 httpd
14458 httpd 1 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14459 httpd 1 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14461 httpd 1 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14464 httpd 1 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14470 httpd 2 0 7788 2632 2360 S 0 0.3 0.5 0:00 httpd
14471 httpd 2 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14472 httpd 3 0 7776 2592 2324 S 0 0.3 0.5 0:00 httpd
14476 httpd 3 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
14495 httpd 11 0 7784 2604 2336 S 0 0.3 0.5 0:00 httpd
14497 httpd 11 0 7780 2600 2332 S 0 0.3 0.5 0:00 httpd
673 root 0 0 216 160 116 S 0 0.1 0.0 7:23 syslogd
14638 httpd 0 0 11220 7860 3924 S 0 0.1 1.5 14:14 httpd
13565 httpd 0 0 9948 5268 3532 S 0 0.1 1.0 0:07 httpd
13764 root 0 0 1472 1456 1132 S 0 0.1 0.2 0:00 sshd
14451 httpd 0 0 7780 2592 2324 S 0 0.1 0.5 0:00 httpd
14455 httpd 0 0 7780 2592 2324 S 0 0.1 0.5 0:00 httpd
14457 httpd 1 0 7780 2600 2332 S 0 0.1 0.5 0:00 httpd

4. your mysql configuration variables located at /etc/my.cnf

UW PICO(tm) 4.2 File: /etc/mysql/my.cnf

# Example mysql config file.

# You can copy this to one of:
# /etc/mysql/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options (in this
# installation this directory is /var/lib/mysql) or
# ~/.my.cnf to set user-specific options.
#
# One can use all long options that the program supports.
# Run the program with --help to get a list of available options

# This will be passed to all mysql clients
[client]
#password = my_password
port = 3306
socket = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

[safe_mysqld]
err-log = /var/log/mysql.err
open-files-limit=8192

[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
log = /var/log/mysql.log
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-locking
#skip-networking
set-variable = key_buffer=16M
set-variable = max_allowed_packet=1M
set-variable = thread_stack=128K
set-variable = max_connections=750
set-variable = myisam_sort_buffer_size=64M
set-variable = join_buffer=3M
set-variable = record_buffer=3M
set-variable = sort_buffer=5M
set-variable = table_cache=1024
set-variable = thread_cache_size=256
set-variable = wait_timeout=600

[mysqldump]
quick
set-variable = max_allowed_packet=1M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition


set-variable = key_buffer=16M
set-variable = sort_buffer=128M
set-variable = read_buffer=16M
set-variable = write_buffer=16M

[i]5. your mysql extended-status output either still telnet as root user type

+--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Aborted_clients | 6358 |
| Aborted_connects | 4 |
| Bytes_received | 628242320 |
| Bytes_sent | 3383710212 |
| Com_admin_commands | 4 |
| Com_alter_table | 6 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 231534 |
| Com_change_master | 0 |
| Com_check | 39 |
| Com_commit | 0 |
| Com_create_db | 0 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 34 |
| Com_delete | 14162 |
| Com_drop_db | 0 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 34 |
| Com_flush | 2 |
| Com_grant | 0 |
| Com_insert | 24028 |
| Com_insert_select | 1504 |
| Com_kill | 0 |
| Com_load | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 3 |
| Com_optimize | 38 |
| Com_purge | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 3500 |
| Com_replace_select | 73 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_rollback | 0 |
| Com_select | 2787466 |
| Com_set_option | 117 |
| Com_show_binlogs | 0 |
| Com_show_create | 117 |
| Com_show_databases | 2 |
| Com_show_fields | 133 |
| Com_show_grants | 0 |
| Com_show_keys | 16 |
| Com_show_logs | 0 |
| Com_show_master_stat | 0 |
| Com_show_open_tables | 0 |
| Com_show_processlist | 0 |
| Com_show_slave_stat | 0 |
| Com_show_status | 125006 |
| Com_show_tables | 47 |
| Com_show_variables | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 0 |
| Com_unlock_tables | 0 |
| Com_update | 452401 |
| Connections | 106406 |
| Created_tmp_disk_tables | 1119 |
| Created_tmp_tables | 80653 |
| Created_tmp_files | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 0 |
| Delayed_errors | 0 |
| Flush_commands | 1 |
| Handler_delete | 60046 |
| Handler_read_first | 255857 |
| Handler_read_key | 24668556 |
| Handler_read_next | 21256598 |
| Handler_read_prev | 0 |
| Handler_read_rnd | 22182644 |
| Handler_read_rnd_next | 591177601 |
| Handler_update | 366954 |
| Handler_write | 5723371 |
| Key_blocks_used | 15582 |
| Key_read_requests | 106935844 |
| Key_reads | 157849 |
| Key_write_requests | 292342 |
| Key_writes | 200880 |
| Max_used_connections | 172 |
| Not_flushed_key_blocks | 0 |
| Not_flushed_delayed_rows | 0 |
| Open_tables | 329 |
| Open_files | 502 |
| Open_streams | 0 |
| Opened_tables | 457 |
| Questions | 3615011 |
| Select_full_join | 1166 |
| Select_full_range_join | 1 |
| Select_range | 448473 |
| Select_range_check | 0 |
| Select_scan | 599882 |
| Slave_running | OFF |
| Slave_open_temp_tables | 0 |
| Slow_launch_threads | 0 |
| Slow_queries | 6 |
| Sort_merge_passes | 0 |
| Sort_range | 439432 |
| Sort_rows | 22792680 |
| Sort_scan | 357711 |
| Table_locks_immediate | 3991098 |
| Table_locks_waited | 2291 |
| Threads_cached | 11 |
| Threads_created | 173 |
| Threads_connected | 162 |
| Threads_running | 1 |
| Uptime | 255567 |
+--------------------------+------------+

or preferred is to installed extended-status output script

http://www.sr20deforum.com/statz.php (web server machine)

6. oh and is your vB the only thing on the server? or other scripts? sites?

There are other sites on here, but none are running any scripts. The machine is basically hosting 2 vBulletin sites.

7. how many average and max concurrent users on your vB forum ?

On average, there are usually 80-100 people between the two forums. It can peak to 120 sometimes.

8. create a file named phpinfo.php and place this code in it and post the url/link to it from your web site

http://ecin.net/phpinfo.php

[eva2000]

from the looks of it, your bottleneck is your raq server, which isn't coping with the load it's getting at peak times...

1. do you have mysql running on both servers ?
2. are both servers in the same datacenter ?