Get "500 - Internal server error" importing large phpbb2 forum

[MagnusT]

Doing a new import from scratch. Deleted the database completely and did a fresh install of the whole vb (again....). Moved the database storage to a faster disk (mirrored SSD) and upped the timeouts even more. Added more memory to PHP (256MB). Right now I'm importing the posts, I run it in 500 posts steps instead of 2000. I also run the impex outside admincp = standalone /impex. Visibly faster than previous import I did the day before yesterday. We'll see how the poll import goes later today. Last time it seemed the poll import ran into problems with all the individual votes table. Just 20000+ rows but possibly some index performance killing join/index/something slowing that particular import down. We'll see.

[MagnusT]

Importing polls just isn't going anywhere. Been doing 500-error -> refresh -> 500-error -> refresh all day without success :-(

[MagnusT]

I have a large phpbb2 board I'm trying to import. It has just over 1.1 million posts. Importing initially went fine using each button in sequence on the impex page. The "import posts" started munching away and ran for 15-20 hours or so, updating regularly showing the progress. I saw it passing 800 thousand posts with no problem. The when I came back later it showed a "500 - Internal server error" page. I did a refresh, waited for a bunch of minutes and got the 500 again. Since I could see in the MySQL Workbench server status screen that when I refreshed the impex page there was work being done by the impex script (SELECTS, UPDATES and so on) I did this refresh a few times.

If I tried to go back to the impex main page /impex/index.php I only got the same 500 error there too after waiting for a while. This seems to be by design though, the index page is = the currently working import page?

Then I shut down the MySQL server. When I did that the impexp webpage showed a list of imported posts and at the end an error message that the MySQL was gone. The interesting part was that the last post listed was the VERY last post in the database. So the script seemed to have imported all posts but had problems with some import tasks that are done after getting all the posts into the database???????

Happy that it had come that far I started MySQL again, did refresh a couple of times with a couple of 500 errors and suddenly I was back at the impex main screen. It had imported the bulk of the posts, there were roughly 1500 errors according to the page.


So now I went to the next step and pushed the button to import polls. And again I get the 500 errors again and again. and again.....



Any hints on what I can do?





------------------------------------------------------------------
Shortlist:

- IIS 7.5 on W2K8R2
- PHP 5.3.8
- MySQL 5.5.19 on another W2K8R2 // memory: 6GB


------------------------------------------------------------------
PHP settings tweaked:

max_execution_time = 900
memory_limit = 128M
post_max_size = 16M




------------------------------------------------------------------
MySQL settings tweaked (most taken from a post in this forum):

[mysqld]
max_connections=250
table_cache=2048
tmp_table_size=131M
thread_cache_size=128
myisam_max_sort_file_size=100G
myisam_sort_buffer_size=261M
key_buffer_size=64M
read_rnd_buffer_size=256K
sort_buffer_size=256K
# skip-innodb
innodb_additional_mem_pool_size=19M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=10M
innodb_buffer_pool_size=864M
innodb_log_file_size=433M
innodb_thread_concurrency=9
join_buffer_size = 4M
read_buffer_size = 4M
sort_buffer_size = 8M
wait_timeout = 180
connect_timeout = 30
max_allowed_packet = 64M
max_connect_errors = 10
query_cache_limit = 2M
query_cache_size = 32M
query_cache_type = 1


[mysqld_safe]
open_files_limit = 8192


[mysqldump]
quick
max_allowed_packet = 16M


[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M



------------------------------------------------------------------

EDIT:

Some additional info: During the running of impex and during the 500 error the rest of the vb site is fully browsable without any errors or problems. As admin I can surf the admin pages and so on. So it's not a general server problem it's just the impex page that gives this error.

The web server log initially looks great with a bunch of status code 200 log rows during all the self refreshing web page impex running. And suddenly there is status 500 and that's it. No additional info than status 500.