3.03 to 3.08 - upgrade19.php crashed - now what?

[RS25.com]

So I posted previously about upgrade19.php taking a long time to process a 1 GB attachment table. However, now the process has died:

Database error in vBulletin :
Link-ID == false, connect failed
mysql error:
mysql error number: 0

Now what do I do?

[RS25.com]

Okay, I guess I panicked a little too soon. I checked into the server and it's doing stuff like crazy, even though the site is smoked:

Mysql Process List
Main >> Mysql >> Show MySQL Processes A {text-decoration: none; color:black } A:hover {text-decoration: underline; color:brown; }
+-------+--------------+-----------+-------------+---------+-------+-------------------+-----------------------------------------------------------------------+| Id | User | Host | db | Command | Time | State | Info |+-------+--------------+-----------+-------------+---------+-------+-------------------+-----------------------------------------------------------------------+| 6647 | eximstats | localhost | eximstats | Sleep | 11 | | || 26530 | rs25_webuser | localhost | rs25_forums | Sleep | 21426 | | || 26907 | rs25_webuser | localhost | rs25_forums | Sleep | 20771 | | || 27779 | rs25_webuser | localhost | rs25_forums | Sleep | 19732 | | || 28844 | rs25_webuser | localhost | rs25_forums | Query | 17658 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 29034 | rs25_webuser | localhost | rs25_forums | Query | 12975 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 29191 | rs25_webuser | localhost | rs25_forums | Query | 16684 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 29249 | rs25_webuser | localhost | rs25_forums | Query | 18287 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29273 | rs25_webuser | localhost | rs25_forums | Query | 18094 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29298 | rs25_webuser | localhost | rs25_forums | Query | 17915 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29311 | rs25_webuser | localhost | rs25_forums | Query | 17735 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29346 | rs25_webuser | localhost | rs25_forums | Query | 17552 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29366 | rs25_webuser | localhost | rs25_forums | Query | 17375 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29382 | rs25_webuser | localhost | rs25_forums | Query | 15480 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 29398 | rs25_webuser | localhost | rs25_forums | Query | 17013 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29409 | rs25_webuser | localhost | rs25_forums | Query | 16836 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29454 | rs25_webuser | localhost | rs25_forums | Query | 16459 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29469 | rs25_webuser | localhost | rs25_forums | Query | 16283 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 29738 | rs25_webuser | localhost | rs25_forums | Query | 16104 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 30176 | rs25_webuser | localhost | rs25_forums | Query | 15925 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 30436 | rs25_webuser | localhost | rs25_forums | Query | 15741 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 30897 | rs25_webuser | localhost | rs25_forums | Query | 15564 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 31277 | rs25_webuser | localhost | rs25_forums | Query | 15383 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 31669 | rs25_webuser | localhost | rs25_forums | Query | 15204 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32253 | rs25_webuser | localhost | rs25_forums | Query | 15024 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32636 | rs25_webuser | localhost | rs25_forums | Query | 14845 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32950 | rs25_webuser | localhost | rs25_forums | Query | 14665 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32969 | rs25_webuser | localhost | rs25_forums | Query | 14485 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32981 | rs25_webuser | localhost | rs25_forums | Query | 14304 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 32992 | rs25_webuser | localhost | rs25_forums | Query | 5467 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 33090 | rs25_webuser | localhost | rs25_forums | Query | 13944 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 33407 | rs25_webuser | localhost | rs25_forums | Query | 11557 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 33681 | rs25_webuser | localhost | rs25_forums | Query | 13584 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 33933 | rs25_webuser | localhost | rs25_forums | Query | 13403 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 34237 | rs25_webuser | localhost | rs25_forums | Query | 8601 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 34558 | rs25_webuser | localhost | rs25_forums | Query | 13043 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 34772 | rs25_webuser | localhost | rs25_forums | Query | 12862 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35051 | rs25_webuser | localhost | rs25_forums | Query | 12683 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35361 | rs25_webuser | localhost | rs25_forums | Query | 12503 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35744 | rs25_webuser | localhost | rs25_forums | Query | 12312 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35935 | rs25_webuser | localhost | rs25_forums | Query | 12127 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35949 | rs25_webuser | localhost | rs25_forums | Query | 11951 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35965 | rs25_webuser | localhost | rs25_forums | Query | 11768 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 35995 | rs25_webuser | localhost | rs25_forums | Query | 11592 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36016 | rs25_webuser | localhost | rs25_forums | Query | 11392 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36051 | rs25_webuser | localhost | rs25_forums | Query | 11218 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36079 | rs25_webuser | localhost | rs25_forums | Query | 11036 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36111 | rs25_webuser | localhost | rs25_forums | Query | 10860 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36131 | rs25_webuser | localhost | rs25_forums | Query | 10674 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36152 | rs25_webuser | localhost | rs25_forums | Query | 10500 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 36435 | rs25_webuser | localhost | rs25_forums | Query | 7168 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 36725 | rs25_webuser | localhost | rs25_forums | Query | 10140 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 37075 | rs25_webuser | localhost | rs25_forums | Query | 9960 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 37452 | rs25_webuser | localhost | rs25_forums | Query | 9779 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 37846 | rs25_webuser | localhost | rs25_forums | Query | 9600 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 38247 | rs25_webuser | localhost | rs25_forums | Query | 9420 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 38645 | rs25_webuser | localhost | rs25_forums | Query | 9240 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 39020 | rs25_webuser | localhost | rs25_forums | Query | 9061 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 39400 | rs25_webuser | localhost | rs25_forums | Query | 8878 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 39760 | rs25_webuser | localhost | rs25_forums | Query | 8700 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40007 | rs25_webuser | localhost | rs25_forums | Query | 8520 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40286 | rs25_webuser | localhost | rs25_forums | Query | 2458 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 40602 | rs25_webuser | localhost | rs25_forums | Query | 8136 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40795 | rs25_webuser | localhost | rs25_forums | Query | 7957 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40821 | rs25_webuser | localhost | rs25_forums | Query | 7774 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40833 | rs25_webuser | localhost | rs25_forums | Query | 7596 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40859 | rs25_webuser | localhost | rs25_forums | Query | 7416 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40887 | rs25_webuser | localhost | rs25_forums | Query | 7235 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 40957 | rs25_webuser | localhost | rs25_forums | Query | 7055 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 41205 | rs25_webuser | localhost | rs25_forums | Query | 6863 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 41529 | rs25_webuser | localhost | rs25_forums | Query | 6685 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 41865 | rs25_webuser | localhost | rs25_forums | Query | 6502 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 41907 | eximstats | localhost | eximstats | Sleep | 6481 | | || 42197 | rs25_webuser | localhost | rs25_forums | Query | 6325 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42524 | rs25_webuser | localhost | rs25_forums | Query | 6141 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42749 | rs25_webuser | localhost | rs25_forums | Query | 5964 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42764 | rs25_webuser | localhost | rs25_forums | Query | 5762 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42798 | rs25_webuser | localhost | rs25_forums | Query | 4133 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 42812 | rs25_webuser | localhost | rs25_forums | Query | 5381 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42840 | rs25_webuser | localhost | rs25_forums | Query | 5208 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42867 | rs25_webuser | localhost | rs25_forums | Query | 5027 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42882 | rs25_webuser | localhost | rs25_forums | Query | 4846 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42910 | rs25_webuser | localhost | rs25_forums | Query | 4668 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42928 | rs25_webuser | localhost | rs25_forums | Query | 4487 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 42937 | rs25_webuser | localhost | rs25_forums | Query | 4301 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 43126 | rs25_webuser | localhost | rs25_forums | Query | 4128 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 43454 | rs25_webuser | localhost | rs25_forums | Query | 3948 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 43766 | rs25_webuser | localhost | rs25_forums | Query | 3768 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 44034 | rs25_webuser | localhost | rs25_forums | Query | 3587 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 44390 | rs25_webuser | localhost | rs25_forums | Query | 3409 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 44786 | rs25_webuser | localhost | rs25_forums | Query | 3227 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 45122 | rs25_webuser | localhost | rs25_forums | Query | 186 | Locked | ALTER TABLE attachment CHANGE thumbnail thumbnail MEDIUMBLOB NOT NULL || 45481 | rs25_webuser | localhost | rs25_forums | Query | 2868 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 45816 | rs25_webuser | localhost | rs25_forums | Query | 2688 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 46113 | rs25_webuser | localhost | rs25_forums | Query | 2508 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 46350 | rs25_webuser | localhost | rs25_forums | Query | 2314 | copy to tmp table | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 46605 | rs25_webuser | localhost | rs25_forums | Query | 2133 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 46836 | rs25_webuser | localhost | rs25_forums | Query | 1950 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 47068 | rs25_webuser | localhost | rs25_forums | Query | 1775 | Locked | ALTER TABLE attachment CHANGE filedata filedata MEDIUMBLOB NOT NULL || 48391 | root | localhost | | Query | 1 | | show processlist || 48392 | rs25_webuser | localhost | | Connect | | login | |+-------+--------------+-----------+-------------+---------+-------+-------------------+-----------------------------------------------------------------------+

Poor server is working overtime with this upgrade:

300 processes: 297 sleeping, 1 running, 1 zombie, 1 stopped
CPU states:  cpu    user    nice  system    irq  softirq  iowait    idle
           total    1.2%    0.0%    5.5%   0.0%     1.1%   91.8%    0.0%
           cpu00    1.7%    0.0%    6.3%   0.0%     0.0%   91.8%    0.0%
           cpu01    0.7%    0.0%    4.7%   0.1%     2.3%   91.8%    0.0%
Mem:  1009240k av,  993448k used,   15792k free,       0k shrd,    4564k buff
                    762828k actv,  144940k in_d,   13808k in_c
[B]Swap: 2040212k av, 2040188k used[/B],      24k free                  312728k cached