Partial vB 3.6 to vB 3.6 Migration

[Christine]

All,

Long story short, I have a split database. 5,000 threads (100,000 posts) from our database were removed. I did a restore onto a new database and restored everything, then deleted all but the 5,000 I need to move back to live.

Everything I am trying is crashing (php limits).

Would it be easier to raise the php limits on size and just let the server churn through them, or do you all have a thread-only vB to vB import that I can try?

[Steve Machol]

You can't do a partial import. What are the exact errors?

[Christine]

They are phpmyadmin errors related to the size of the file. I ran a script to load the thread table from CVS into the format of
INSERT INTO thread (threadid, title, lastpost, forumid, pollid, open, replycount, postusername, postuserid, lastposter, dateline, views, iconid, notes, visible, sticky, votenum, votetotal, attach, firstpostid, similar, hiddencount, deletedcount, lastpostid) VALUES ('178463', 'Christmas Day', '1102576013', '37', '0', '0', '4', 'Chaplain', '14603', 'Christine', '1102573118', '52', '4', ' Thread Closed by Christine on December 9th, 2004 at 02:06 AM.',

Then split it in half in notepad to get under 2mb. That ran, but I have no clue what to do with the 100,000 posts I have out there that need to be brought back in. That could take weeks (literally).

[Steve Machol]

This may be a server time and/or memory limit. To temporarily up your limits edit your includes/config.php file and add these lines right under the <?php line:

ini_set('memory_limit', 32 * 1024 * 1024);
ini_set("max_execution_time", "800");

[Christine]

Will do Steve -- much thanks.

How large of a file will that allow me to load?

[Steve Machol]

File? What file? I thought you were doing an import.

[Christine]

Yes, importing a .sql file via phpmyadmin.

[Steve Machol]

That's not the same as using the Impex importer which is the forum you posted in. ;)

The only reliable method of restoring a database is with shell access via telnet or ssh. This is because restoring with a PHP script like that in phpMyAdmin can result in PHP timeout errors and an incomplete database. Please see the instructions here:

http://www.vbulletin.com/docs/html/maintenance_ssh_restore

Note: Some people have reported success with these scripts:

MySQLDumper:
http://www.mysqldumper.de/en/index.php

MySQLHotxcopy:
http://www.vbulletin.com/forum/showthread.php?t=134821&highlight=mysqlhotcopy

Bigdump (restore only):
http://www.ozerov.de/bigdump.php

[Christine]

Steve,

I think we are talking about 2 different things. I am asking in the Import forum because that is what I want to do. I need to import 5,000 threads from one vBulletin database into an existing vBulletin database (these threads were deleted out of the database and I need to restore them back into it, not restore/overwrite the whole thing).

I am asking if your import scripts will allow me to do that -- merge in 100,000 posts back to a database that they were originally in.

WHen you said no, that left my only option to try to load them manually via SQL insert statements.

Is it possible to IMPEX these 5.000 threads (100,000 posts) back into the database that they were deleted out of?

[Steve Machol]

No, Impex does not import data from a backup to a database. You need to get that data into the database first.

[Christine]

Steve,

I am not understanding why you and I are seemingly on two different continents with this conversation. LOL! :D

It is in a database. Per my opening post, I have 2 vB 3.6 databases. One is my live database (database 1), that is missing 5,000 threads, 100,000 posts.

The other (database 2) is that SAME database but is ONLY the missing 5,000 threads, 100,000 posts from database 1 as I purged everything but the missing threads/posts after I restored it a few days ago. The reason I did this was to have a clean database with ONLY the missing thread/post data and no other.

I need to import the threads and posts (and relevant tables like editlog) from database 2 back into database 1 where they started from.

I tried it manually with sql scripts and it is crashing -- so I opened this thread to ask if the IMPEX system could help me.

I am still not clear on the original question I came here to ask: Can IMPEX bring in thread and post data from one vB 3.6 into another vB 3.6?

[Steve Machol]

Steve,

I am not understanding why you and I are seemingly on two different continents with this conversation. LOL! :D

It is in a database.


Yes, importing a .sql file via phpmyadmin.

Frankly I'm confused. Either you are importing an .sql backup into a database or you are using Impex to import data from an existing db into vB.

[Christine]

You can't do a partial import. What are the exact errors? Steve,

It *looks* here like you are answering my question by saying that I can't do a partial import -- so we switched to talking about how I was trying to load up, the errors on which prompted me to ask about the import functionality.


That's not the same as using the Impex importer which is the forum you posted in. I am quite aware of that and am not looking for any assistance with running SQL scripts. I was just curious as to the file size permitted on the config.php mod you noted.




The question remains -- Can I or can I not use the IMPEX technology to restore 5,000 threads from one vB 3.6 database into another vB 3.6 database given that these 5,000 threads *used* to belong in said [original] database, they have been restored into a new database where they are on their own (all other threads have been deleted). There is no duplication of threads and no orphaned threads -- I just need to plug them back into where they used to be?

Would it be better if I just asked for IMPEX and figured it out on my own as opposed to asking if it will do what I need?

[Steve Machol]

If those 5000 threads are the only threads in that database then you should be able to import them. However you would first need to import the users and forums. These are required prior to importing the threads. After that you will of course need to import the posts for those threads as well.


I just need to plug them back into where they used to be?
That's not possible with Impex. Importing will assign new thread and postids as well as new forums for these threads. You can then move them to other forums after importing if you want.

[Christine]

Thanks Steve,

This is what I was looking for. :) I started my vB in 2001 from scratch and never had to use IMPEX, so wasn't sure what all it did.

I would like to take a look to see what I can do with this.

Where can I d/l this, or how do I go about requesting a copy?

[Steve Machol]

Impex is available for download in the Members Area.

[antiekeradio]

can't you use bigdump to restore the .sql file to the database? if there are no duplicate ID's etc, this should go on without overwriting/messing up anything.

be sure to try it out in a test environment first though...

[Christine]

Impex is available for download in the Members Area.Thanks!


can't you use bigdump to restore the .sql file to the database? if there are no duplicate ID's etc, this should go on without overwriting/messing up anything.

be sure to try it out in a test environment first though...Interesting. You are referring to this? http://www.ozerov.de/bigdump.php

It looks like that utility requires a DROP TABLE statement in the sql script. I am not looking to drop/replace the table -- I just need to merge the old threads back into it.

Did I miss that on that page?