Upgraded to 3.6.2, both forums now giving "blocked because of many connections" error [Archive]

Wings900

Tue 10th Oct '06, 12:50pm

Upgraded our two installations of vb to 3.6.2 yesterday. Overnight we were flooded with over 400 emails with the following:

mysql_connect(): Host 'localhost' is blocked because of many connection errors. Unblock with 'mysqladmin flush-hosts'
/home/diecastw/public_html/w900/vb/includes/class_core.php on line 271

we increased max_connect_errors from 10 to 500, only stopped the bleeding for about an hour. Now we rebooted, and set max_connect_errors=10000

Something is wrong here, we ran fine until the upgrade yesteday. Our /etc/my.cnf is below:

[mysqld]
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time= 3
skip-locking
skip-innodb
query_cache_limit=1M
query_cache_size=32M
query_cache_type=1
max_connections = 500
interactive_timeout=100
wait_timeout=100
connect_timeout=10
thread_cache_size=128
key_buffer=16M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
join_buffer=1M
read_buffer=1M
sort_buffer=1M
max_allowed_packet=16M
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
max_connect_errors=10000
thread_concurrency=1
myisam_sort_buffer_size=64M
server-id=1
table_cache = 1024
tmp_table_size = 64M
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M
[safe_mysqld]
err-log=/var/log/mysql/mysqld.log
open_files_limit=8192
[mysql]
no-auto-rehash
[isamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M
[safe_mysqld]
err-log=/var/log/mysql/mysqld.log
open_files_limit=8192
[mysql]
no-auto-rehash
[isamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M
[myisamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M
[mysqlhotcopy]
interactive-timeout

Ideas on what is going on? Thanks!

Steve Machol

Tue 10th Oct '06, 1:32pm

Try increasing max_connections.

Wings900

Tue 10th Oct '06, 1:57pm

Try increasing max_connections.

No go, we are getting bombarded with errors every few seconds..

-t

Steve Machol

Tue 10th Oct '06, 2:01pm

What did you change it to and did you restart MySQL? How many connections do you have?

Wings900

Tue 10th Oct '06, 2:05pm

Steve Machol

Tue 10th Oct '06, 2:07pm

You have disabled persistent connections in config.php, right?

If you are getting these errors then the max_connections value is not high enough. Try increasing it again.

Also since this is a server issue I'm moving this to the Server Configuration forum.

Wings900

Tue 10th Oct '06, 2:12pm

You have disabled persistent connections in config.php, right?

If you are getting these errors then the max_connections value is not high enough. Try increasing it again.

Also since this is a server issue I'm moving this to the Server Configuration forum.

$config['MasterServer']['usepconnect'] = 0;

Setting
max_connections = 10000

Restarting mysql..

Can you say if anything has changed in 3.6.2 versus 3.6.1 in terms of code changes thay may be causing this?

-t

Steve Machol

Tue 10th Oct '06, 2:15pm

I'm not aware of any code changes that would cause this.

Wings900

Tue 10th Oct '06, 4:04pm

Server just went out again,

mysql_connect(): Lost connection to MySQL server during query /home/diecastw/public_html/w900/vb/includes/class_core.php on line 271

Got two of those and then

mysql_connect(): Host 'localhost' is blocked because of many connection errors. Unblock with 'mysqladmin flush-hosts'
/home/diecastw/public_html/dac/includes/class_core.php on line 271

This is frustrating as all heck..

eva2000

Thu 12th Oct '06, 1:10pm

sounds like TCP/IP network issues on your server http://dev.mysql.com/doc/refman/4.1/en/blocked-host.html and http://dev.mysql.com/doc/refman/4.1/en/communication-errors.html

Can you post all info asked at http://www.vbulletin.com/forum/showthread.php?t=70117

Wings900

Fri 13th Oct '06, 12:15am

1. is this on dedicated or shared virual server
Dedicated

2. your server specs. For example:
cpu speed/type single or dual cpus): Single 2.4ghz Celeron
how much memory installed: 1GB
hard drive type/configuration: Dual IDE, one mounted for system, other as backup, not configured for RAID
linux distributor or windows version: RH Enterprise3 - Kernal 2.6.17.3
Apache/IIS version: apache 1.3.34
PHP version: php 4.4.2
MySQL version: mysql 4.0.27-standard

3. if you use mysql 4.x instead of mysql 3.23.x, do you have any innodb type databases/tables on your server ?
Not that we know of

4. if possible how mysql was compiled/installed
W/cpanel

5. your top stats
http://www.wings900.com/opt/top.txt

6. your mysql configuration variables
http://www.wings900.com/opt/mysql.txt

7. your mysql extended-status output
http://www.wings900.com/opt/mysqlextended.txt
http://www.wings900.com/opt/mysqlreport.txt

8. is your vB the only thing on the server? or other scripts & sites which utilise php and mysql?
There are two instances of VB, postnuke, photopost and custom written software

9. how many average and max concurrent users on your vB forum ? and what your cookie timeout is ?
Average between 50-100, cookie time out is 900

10. create a file named phpinfo.php and place this code in it and post the url/link to it from your web site
http://www.wings900.com/opt/phpinfo.php

11. if you run Apache and you have your own dedicated server or access to your httpd.conf (apache configuration file) can you post the values you have set for the following :
KeepAlive OFF
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 10
MaxSpareServers 15
StartServers 10
MaxClients 200

12. what version of vB are you running ?
3.6.2 (both installs)

13. MISC
We have Eaccelerator installed

14. Post output from these 3 commands

uname -a
Linux tucker.diecastwings.net 2.6.17.3 #1 Sun Jul 2 12:34:28 EDT 2006 i686 i686 i386 GNU/Linux

ulimit -aH
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 14335
virtual memory (kbytes, -v) unlimited

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Celeron(R) CPU 2.40GHz
stepping : 9
cpu MHz : 2394.138
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe up cid xtpr
bogomips : 4792.01

eva2000

Sun 15th Oct '06, 11:12am

Try these changes although i can't pin point any configuration issue for php/mysql etc that could cause this besides actual network issue at your server/web host end.

Can you please install mysqlreport as well from http://www.vbulletin.com/forum/showthread.php?t=175177 and post the output you get preferably before you try any of the below suggested changes.

1. Upgrade to Apache 1.3.37
2. Upgrade PHP to 4.4.4 and/or recompile PHP 4.4.4 after upgrading MySQL
3. Edit /etc/my.cnf and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards

[mysqld]
safe-show-database
skip-innodb
max_connections = 500
key_buffer = 48M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 1500
thread_cache_size = 384
wait_timeout = 25
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
max_connect_errors = 10
read_rnd_buffer_size = 524288
bulk_insert_buffer_size = 8M
query_cache_limit = 5M
query_cache_size = 80M
query_cache_type = 1
query_prealloc_size = 163840
query_alloc_block_size = 32768

[mysqld_safe]
err-log=/var/log/mysql/mysqld.log
open_files_limit = 8192

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M

[mysqlhotcopy]
interactive-timeout

4. Install APC Cache lastest version from http://pecl.php.net/package/APC. Read install guide at http://www.vbulletin.com/forum/showthread.php?t=165367 - please remove Zend Optimizer from php.ini before installing APC Cache

5. Edit httpd.conf values from

KeepAlive OFF
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 10
MaxSpareServers 15
StartServers 10
MaxClients 200

to

KeepAlive OFF
MaxKeepAliveRequests 100
KeepAliveTimeout 5
MinSpareServers 10
MaxSpareServers 15
StartServers 10
MaxClients 150

restart apache

6. If you just upgraded to vB 3.5.x/3.6.x try to disable these 4 options:

Admin CP -> vBulletin Options -> Forums Home Page Options -> Display Logged in Users?

Admin CP -> vBulletin Options -> Forum Display Options (forumdisplay) -> Show Users Browsing Forums

Admin CP -> vBulletin Options -> Thread Display Options -> Show Users Browsing Thread

Admin CP -> vBulletin Options -> Message Searching Options -> Automatic Similar Thread search

Wings900

Mon 16th Oct '06, 4:36pm

Can you please install mysqlreport as well from http://www.vbulletin.com/forum/showthread.php?t=175177 and post the output you get preferably before you try any of the below suggested changes.

This was already done and was in my original response:
http://www.wings900.com/opt/mysqlreport.txt

-t

Wings900

Mon 16th Oct '06, 5:00pm

4. Install APC Cache lastest version from http://pecl.php.net/package/APC. Read install guide at http://www.vbulletin.com/forum/showthread.php?t=165367 - please remove Zend Optimizer from php.ini before installing APC Cache

Completed this, but did not install. We are running Eaccelerator, do you recommend APC over eac?

-t

eva2000

Wed 18th Oct '06, 3:14am

Completed this, but did not install. We are running Eaccelerator, do you recommend APC over eac?

-t
yes if you use PHP 4.4.4 if you use PHP 5.1.6 probably stick with eaccelerator