Importing 4.6 million phpbb posts

[Janm]

Hi,

Yesterday I started with a test conversion from phpbb3 to vb. I have a rather large forum with over 4.600.000 posts and the conversion stopped at 500.000 posts probably 5 minutes after I went to bed.

This morning I reloaded the page, it asked me my login details, I clicked on Import again and it started where it left off.

My question is, is this okay? I'm not missing any posts?

Furthermore I'd like to ask if there are things I can do to speed up this process or things I have to think about regarding the amount of posts.

Other tips and tricks are more than welcome :)

Thanks,

Jan
The Netherlands.

[Floris]

Yeah, a refresh should just pick up where it stopped.

[Janm]

Thanks!

But how come it stops after about 12 hours of conversion? Does it have something to do with the time you can stay logged in?

Two nights on a row it happened I start the conversion in the morning it runs all day and around midnight it stops. I have to refresh and then it runs again and it runs all night.

Because of the huge amount of posts I have to convert it would be better for me if the script would just keep on running.

I started last friday afternoon and it's now monday morning and I converted 2.7 million posts. I missed two nights of conversion so I think it takes me 4 days to convert all posts. I need the script to run 96 hours an a row. Can this be done?

And does anyone know how long it takes to rebuild search with 4.6 million posts?

Jan
The Nerherlands

[Floris]

You are limited to the timeout settings in the browser, web server, php values, mysql, etc .. if something takes too long (for example mysql reloads or a socket hangs, etc) it can timeout.

[Janm]

Thank you Floris! I also contacted my host and probably this is due to a apache restart every night at 00:10 hours.

About the search .. how long does it take to rebuild search for 4.6 million posts?

Jan
The Netherlands

[Floris]

A long time ..I recommend looking into the donotupload/ folder and specifically the searchshell.php file :) Which is a SHELL script, helping you bypass the browser limits

[Janm]

Thanks Floris will do that!

I am at 4 million at the moment so I'm almost there :)

After that I'll run the database cleanup from the link in acp. Do I have to run another cleaner?

And I installed VB with an admin account but phpbb3 also imports two admins, can I delete the admin account created by me to install VB?

Jan
The Netherlands

[Floris]

You can merge user accounts if they're not properly merged. Once the whole import is done I'd go about clean up the usergroups, forums, accounts, and check all the permissions. After running most of the update counters.

[Janm]

Hi Floris,

This morning he was finished. Converting 4.6 million posts takes 109 hours.
I rebuilt cache threads and forums and I'm rebuilding search database at the moment, which goes rather fast I have to admit.

I would like to make sure all posts are clean so I tried to use cleaner.php from the tools folder of impex but that didn't work. How can I get to the cleaner.php and how can I excute it?

Jan