Merging two databases

[Wattie]

I have 2 separate vbulletin forums but want to merge them into one. I only want the basic user info (name, email, posts etc) from the one I will be merging but would like to be able to keep the forums and posts. Does anyone know if this is possible with Impex? Cheers

[Colin F]

That's possible with ImpEx, you'd simply only run the modules you want.

[Wattie]

Thanks. Will it overwrite existing forums which have the same forum id number?

[Colin F]

No, all the imported data will receive new unique ids.

[Wattie]

well i used impex and it completely overwrote the forum! I now have lost nearly 3500 users and all the threads etc.

[Steve Machol]

That's not possible. You had to have done something else. Impex does not overwrite or remove your existing users.

[Wattie]

I ran IMPEX to merge the two forums together and it completely overwrote the existing one with the forum I was importing. I kid you not. It overwrote the users, forum information, threads, etc. I looked in the database to see if they were there and couldnt find them there either. To say I am hacked off at the moment is a huge understatement.

[Wattie]

Just checked again and it DID wipe everything from my existing database. All I did was run the Impex script. The server kept timing out when I tried to backup the database, but having had no problems when importing from phpbb2 to vb in the past I thought I'd chance it. Wish I hadnt bothered now as I have a huge job on my hands reparing this mess.

[Wattie]

And no response is forthcoming! Some of the old forums are still there (I have found) but I cannot access them, the maintenance script doesn't reactivate them etc. ALL thready from the last 3 months have been lost and I am furious that what should have been a simple operation has proved so costly. It will take me DAYS to repair the damage this has caused. THANKS! I suggest the IMPEX script is looked at very closely. I followed the instructions in your online manual exactly and it has got me in a right mess.

[Wattie]

i have found that there are a couple of surviving forums in the database. However, they do not appear in my forum index and when I type the unique id for them it says I havent permission. I have tried creating a new forum and changing the id to it but that doesnt work, so how can I get them back into the main forum?

[Wattie]

I have been struggling with this for over an hour now and apart from 'you must have done something else', support here has been non-existent. I followed your instructions to the letter and have got in a right mess. At least you could offer some assistance.

[Colin F]

Please be advised that the main method of support is the ticket system, if you have something urgent, please use that.

I'm not sure what I should tell you. The ImpEx script will never overwrite default data, the only way that would happen is if you already imported once (possibly also from a different forum), and didn't remove the importids.

[Wattie]

i had initially imported it from a vb3 board to 3.5. It didnt say anywhere in the documentation that there was any issue with import ids! I changed the impex config file with the new data if that is what you mean so I dont know what else I could have changed.

[Colin F]

The problem is, you can run a module multiple times. When you run it a second time, ImpEx deletes the previous posts that have an importid.
When you're all the way done with an import, it says to run the option to delete these importids.
Because you didn't do this, it assumed you were just rerunning a module, thus it overwrote the existing data that had importids.

[Steve Machol]

Also this wouldn't have happened if you followed the 'After the Import' instructions here:

http://www.vbulletin.com/docs/html/impex_cleanup?manualversion=30501500

[Jerry]

.......I followed your instructions to the letter and have got in a right mess. ........

You followed the multi import instructions to the letter ?

You finalised the first import before doing the second and you took a back up before doing anything ?


but having had no problems when importing from phpbb2 to vb in the past I thought I'd chance it.

http://www.vbulletin.com/docs/html/impex_multi_import


You MUST remove the import id's before performing a 2nd import, this is because ImpEx will not know what is original imported data and what is just imported etc.

If you had done that there is no way that ImpEx could delete the existing data. I'd never suggest "chancing an import", especially a merge import.


It will take me DAYS to repair the damage this has caused. THANKS! I suggest the IMPEX script is looked at very closely. I followed the instructions in your online manual exactly and it has got me in a right mess.

The first instruction or any import is make a back up and test the import, do not run it on a live database.

http://www.vbulletin.com/forum/showthread.php?t=116398

ImpEx is looked at very closely, constantly.

[Wattie]

Also this wouldn't have happened if you followed the 'After the Import' instructions here:

http://www.vbulletin.com/docs/html/impex_cleanup?manualversion=30501500I DID follow those instructions and I dont remember ANY option to delete the import ids after I had upgraded from 3.0 to 3.5
I also have to say that as I have paid for the product and rarely asked for support, I do not appreciate being treated like a thick schoolboy when asking for assistance. The fact is that I asked for help on this matter before embarking on the merge using IMPEX and having not recieved ANY warnings or sufficient support, it caused a lot of problems. Your instructions did not, as far as I could see, include any warnings.
To say I am disappointed with this matter would be a huge understatement.
On my other thread subsequent to this episode I simply asked for help in ensuring it wouldnt happen again. All I get is 'if you'd done this' or 'should have done that'. Would it really hurt to give me simple straight forward instructions to achieve something you claim is very straight forward. Or is that asking too much?

[Steve Machol]

The Impex import instructions are available here:

http://www.vbulletin.com/docs/html/impex?manualversion=30501500

Also please view the 'After the Import' instructions here:

http://www.vbulletin.com/docs/html/impex_cleanup?manualversion=30501500

If you think there is an error or specific omission in the instructions please let us know and we will correct it.

[Wattie]

As I have already told you, I followed all of those instructions to the letter along with the 'after the install' instructions. The IMPEX installation was a new download which I placed on my server the night before doing the new merge. In reference to my messages, I am not wanting to complain or apportion blame, I simply want to know how to merge two forums together without the same thing happening again.
I asked beforehand whether the IMPEX script would overwrite the data, and as you can see from this thread, I was told
No, all the imported data will receive new unique ids.Obviously this did not happen as my forum was overwritten completely and if there is the possibility of it happening if I had or hadn't done something, surely you were remiss in not making me aware of it. I fail to see what else I could have done to ensure the problems I encountered did not occur. So, please can you tell me how to ensure this does not happen again as I still would like to merge the two forums together. Alternatively can you tell me if it is possible to use the same user database as this would accomplish the same thing!

[Steve Machol]

I'm not trying to be difficult but I don't know what else I can tell you that is different than what we have already posted. This simply does not happen when Impex is run correctly. Please reread Jerry's post (#30.) If Impex was doing this then our forums would be full of people complaining about this.

If using Impex is not acceptable to you, then at this time your only other option would be to find someone to write a custom import script for you. You can try posting in the Service Requests forum at vbulletin.org for help with this.

I'm sorry I can't give you the answer that you want. Maybe Jerry will be better able to explain this when he is available.

[Wattie]

I am also not trying to be difficult or come across as a whinger. The fact is though that you do not seem to take into account that not everyone is as computer savvy as you are. Perhaps you take your customers' knowledge for granted in some aspects. Surely if I asked about it being overwritten, the correct response would have been; that will not happen unless you have previously imported a board and not removed the ids.' This would have given a warning that there was a chance. As it was, the information was not, in my opinion, very clear. I have looked at your 'multi import' instructions and to be honest, when I ran impex, it asked for my customer number and then ran the application. I do not remember seeing any other options such as 'clean database'. Maybe I missed them but I certainly do not remember that as an option. Also, not wishing to be a peddant, but your instructions say 'To remove the import id's use the clean database found at the top of the ImpEx page, as show in the attached images.' The images in question do not appear to be on the page unless the are attached somewhere else. So is there a way to run the clean database script now without it trying to overwrite the forum again?

I have to say though that in contrast to how I am feeling with this discussion, your support system after the problem emerged was very good and effective.

[Jerry]

So is there a way to run the clean database script now without it trying to overwrite the forum again?

Yes, after each import before performing the next import and when you are sure its ok, go to the help page (at the top of any ImpEx page) and from there you can remove the import id's from the database.

That finalizes an import and allows you do do consecutive imports with out effecting the data, i.e.


Empty board.
Do import 1.
Remove import ids, import is finalized now.

Do import 2, data from import 1 has no import id's, new data from import 2 has import id's so ImpEx know what it just imported and what is existing data that it's not allowed to delete or modify.
Remove import ids, import is finalized now.
Do import X
Remove import ids, import is finalized now.
.... repeat etc

[LeftCoastBias]

... my admin at my site is a mysql database admin professionally (day job; he moonlights as a freak of nature on my site).

is this something he should be able to do in between eating slices of toast without spilling jelly all over his keyboard and/or destroying our entire forum?


i'm not sure i'm willing to try this on my own; but another forum we are friendly with has come to ask us to "annex" them. we'd be more than happy to bring their members and data into our own- but our main priority is, of course, the safekeeping of our forum.

[Jerry]

... my admin at my site is a mysql database admin professionally (day job; he moonlights as a freak of nature on my site).

There is more than one of me ....... I didn't think they gave that many passes to get away from our planet !!


is this something he should be able to do in between eating slices of toast without spilling jelly all over his keyboard and/or destroying our entire forum?

Yup, its an easy process and for an admin, and one who understands config files and databases even more so. If the instructions are followed and its tested on a back up of the forum first, you can only mess up the test install the live forum is separate and immune to jelly and flying toast attacks.


i'm not sure i'm willing to try this on my own; but another forum we are friendly with has come to ask us to "annex" them. we'd be more than happy to bring their members and data into our own- but our main priority is, of course, the safekeeping of our forum.

This is what ImpEx is for, if you are importing another board into an existing vBulletin that hasn't had an import done before, its the simplest merge import as ImpEx will know what is the original board and what is the new data so if modules are re-run it will only clear out the data that it imported, just don't merge any users till you are happy with what is going on.

[AllAboutAV]

Hey guys.

I am new to vBulletin, and have been running 3.5.4 successfully for about a week. I converted from a phpbb2 system. The initial Impex install went great!

Well, my problem is similar to those above, so I am posting a question here first. Basically , I am not sure that I completely remove ALL of the "imported" tags correctly, (I didn't realize at the time there was a script for it - der) and when I did a second import to bring in some really old posts I found in a backup (that I that had been lost years ago) I "lost" the data that had been added since the vBulletin Install. The data is actually all there in the tabels. Every bit of it, but I am not that database savvy to know how to make it show back up. I have all of the olds posts showing up great, and I have a backup of the entire database (WITH the current issue, but all of the data).

I am a quick study and would be appreciative of any help offered. I would like to make the repairs myself as well, so I can learn from my mistake. :)

[LeftCoastBias]

There is more than one of me ....... I didn't think they gave that many passes to get away from our planet !!



Yup, its an easy process and for an admin, and one who understands config files and databases even more so. If the instructions are followed and its tested on a back up of the forum first, you can only mess up the test install the live forum is separate and immune to jelly and flying toast attacks.



This is what ImpEx is for, if you are importing another board into an existing vBulletin that hasn't had an import done before, its the simplest merge import as ImpEx will know what is the original board and what is the new data so if modules are re-run it will only clear out the data that it imported, just don't merge any users till you are happy with what is going on.
awesome. thanks for the info.


i'll let you know if any administrators are harmed during production.