UBB.threads™ 6.4 to vBullletin 3.0.3

[klaus]

I today tried again to import from UBB.threads™ 6.4 to vBulletin 3.0.3 with only mixed results. My UBB board is not small with 105.000 posts, 11.000 threads and 2.100 users and the import process takes a couple of hours.

The below Items are the ones that don't import right and need to be imported for me to switch and use vBulletin. I struggled importing with non working vBulletin V2.0 import scripts and are fed up with paying for software that doesn't work (for me).

HELP!



1.User

- Signature - Not Imported

- Usergroup - Not Imported

- Avatar - Not Imported

- User Profile Fields - Not Imported

- Private messages - Not showing up in CP, yet PMs can be seen in the mysql db.

- Home Page URLs now have my sites URL as a prefix. Example: my site is www.a.com (http://www.vbulletin.com/forum/www.a.com), new Home Page URL in user profile now is: www.a.com/www.b.com (http://www.vbulletin.com/forum/www.a.com/www.b.com) when before it was just www.b.com (http://www.vbulletin.com/forum/www.b.com)



2.Polls

- Not Imported

3.Attachments

- All previous attachments disappeared! Under UBB attachments are stored outside the Mysql db.

4.Usergroups

- The UBB groups get created but all imported user simply become "Registered User". Not all of my old users are registered users plus I have a couple of usergroups with certain access masks.

- All forums are "registered User" readable even though in UBB some are private(I dont mind changing the Forum permissions manually).

5.Calendar

- Calendars - Not Imported







When running the first step in Impex "check and update database" I get the following result: (note the Poll details)

--

Import / Export

Altering tables

ImpEx will now Alter the tables in the vB database to include import id numbers.

This is needed during the import process for maintaining references between the tables during an import.If you have large tables (i.e. lots of posts) this can take some time.

They will also be left after the import if you need to link back to the origional vB userid.



moderator - importmoderatorid OK

usergroup - importusergroupid OK

ranks - importrankid OK

poll - importpollid OK

forum - importforumid OK

forum - importcategoryid OK

user - importuserid OK

style - importstyleid OK

thread - importthreadid OK

post - importthreadid OK

thread - importforumid OK

smilie - importsmilieid OK

pmtext - importpmid OK

avatar - importavatarid OK

customavatar - importcustomavatarid OK

customprofilepic - importcustomprofilepicid OK

post - importpostid OK

w3t_AddressBook found.

w3t_Banned found.

w3t_Boards found.

w3t_Cache NOT found or a non standard table.

w3t_Calendar NOT found or a non standard table.

w3t_Category found.

w3t_DisplayNames found.

w3t_Favorites found.

w3t_Graemlins found.

w3t_Groups found.

w3t_Languages found.

w3t_Last found.

w3t_Mailer found.

w3t_Messages found.

w3t_ModNotify found.

w3t_Moderators found.

w3t_Online found.

w3t_PollMain NOT found or a non standard table.

w3t_PollOptions NOT found or a non standard table.

w3t_PollQuestions NOT found or a non standard table.

w3t_PollVotes NOT found or a non standard table.

w3t_Posts found.

w3t_Queries NOT found or a non standard table.

w3t_Ratings found.

w3t_Subscribe found.

w3t_Users found.

w3t_Version NOT found or a non standard table.

vBulletin v3.0.3, Copyright ©2000-2004, Jelsoft Enterprises Ltd.

--

[Jerry]

Quite a lot going on there , though I before I answer and check it all.

Are you using the latest version of ImpEx ?

The poll table checking and the usergroup errors (U_Groups) leads me to believe you might not have the most up to date version ?

I have checked the ImpEx download in the members area and rebuilt the ubb.threads importer from CVS just to make sure.

[coloradok5]

Klaus
I just did this Import as a test, looks like the users and the user groups, posts and reply's imported OK and thats all I really care about. The polls and profile (correctly) as well as signitures did not but that is something users can redo on their own, most of my polls are about silly stuff anyway so no big deal there.
I plan on keeping my old forum up for a while after the import so users can go back and get something if it didn't import, I think the hassle is well worth the reward, VB blows away UBBT by far.

[Jerry]

This is rather strange as I re-wrote the Polls module for ubb a few weeks ago I think it was.

[klaus]

my memebership expired a couple of days ago...so i'll front the $30.00 and make sure i have the most recent Impex and try again.

Polls are not important to me as well. They are usualy about a current subject matter and then turn into old news on our site...so old polls dont need to be imported.

The Profile fields are very important as they are required fields in my old board. Same with usergroups to user associations. I would otherwise have to do it manualy. Not realistic with 2000+ users.

the cleaner.php works awesome in cleaning up messages. All URLs had a wrong prefix in them and the cleaner.php changed all of them in no time.

I assume cleaner.php only cleans the posts? Could I use it to clean the wrong homepage URLs ?


next step on my plate before I post progress:

- Renew vb memebership
- Download latest Impex
- Run Import again.

[Jerry]

my memebership expired a couple of days ago...so i'll front the $30.00 and make sure i have the most recent Impex and try again.

Polls are not important to me as well. They are usualy about a current subject matter and then turn into old news on our site...so old polls dont need to be imported.

Either way i'd like to see what is going on there, as I've written the module to deal with 3 versions so I'm intrested as to why it isn't working.


The Profile fields are very important as they are required fields in my old board. Same with usergroups to user associations. I would otherwise have to do it manualy. Not realistic with 2000+ users.

What kind of fields ? location, occupation, etc ? or custom ones ?


the cleaner.php works awesome in cleaning up messages. All URLs had a wrong prefix in them and the cleaner.php changed all of them in no time.

I assume cleaner.php only cleans the posts? Could I use it to clean the wrong homepage URLs ?

It cleans posts, signatures and thread titles.

Do you mean the homepage for the user profile ?

next step on my plate before I post progress:



- Renew vb memebership
- Download latest Impex
- Run Import again.

Email support@vbulletin.com and I can help with getting it spot on ASAP.

[klaus]

downloaded the current version of Impex (11-30-04), renewed my license (as I should have anyways) and I noticed major improvements:

- Everything imports a lot faster and the memory error messages (see other peoples complaints/threads) are gone. I can now import with the default values.
- Polls get imported, not sure if propper as the import still runs right this minute.
- Attachments get imported but it hangs on one particular attachment file. the default attchment limits are set rather low and I am sure some of my old attachments excite the limits. Should I increase attachment limits before importing? Is there a way to restart just one import step rather then all of them (time).

Full progress report later

[Jerry]

- Attachments get imported but it hangs on one particular attachment file. the default attchment limits are set rather low and I am sure some of my old attachments excite the limits. Should I increase attachment limits before importing? Is there a way to restart just one import step rather then all of them (time).

Full progress report later

Configure your board to use the file system for attachments opposed to the database.

Then run the attachmenets import.

You can re-run modules, they clean up after themselves, just click the button where it says "REDO".

[klaus]

From my initial complaints a lot of them have been knocked out today.
Awesome. Thank you. Turned out that I was using an older Impex.

Things that are still not working right:

1.User
- Avatar - Not Imported (not important)
- Birthday - Not Imported (table: Users , field: U_Birthday (mm/dd/yyyy), U_ShowBday (0 or 1)
- User Profile Fields - Not Imported, Under UBB I had "Location, Hobbies(Interests), Bio, Occupation and then 4 extra fields. (table: Users, fields: U_Bio, U_Occupation, U_Hobbies, U_Location, U_Extra1 .... U_Extra5)
- Home Page URLs now have my sites URL as a prefix. Example: my site is www.a.com (http://www.vbulletin.com/forum/www.a.com), new Home Page URL in user profile now is: www.a.com/www.b.com (http://www.vbulletin.com/forum/www.a.com/www.b.com) when before it was just www.b.com (http://www.vbulletin.com/forum/www.b.com)

5.Calendar
- Calendar - Not Imported (not important to me, I have 13 private entries and 1 public)

[Jerry]

User - Avatar

Done.

User - Birthday

Done.
ShowBday is part of the usergroup settings in vBulletin.

User - Profile Fields

U_Occupation, U_Hobbies, U_Location, done.
U_Extra1 .... U_Extra5 done,U_Bio, imported as custom fields.

User - Homepage

What is in U_Homepage is imported, can you check the source database to ensure the source data isn't corrupt. Either way, I can give you the code to use cleaner.php to sort this out if you need it.

Calendar

No current plans to import.

All updates are in the 1.33 download.

[klaus]

Good news, will download and install the new version of Impex later today and givethings a shot.

In regards to the Homepage:

Under UBB in the table Users the field U_Homepage has either NULL entries or the website with out the http : // in front of it Example: www . google . com (no spaces)

if you can make cleaner do the cleanup then that will work for me. I use cleaner to clean up all kinds of URLs for embeded attachments/images/smilies etc

[klaus]

Just finished another import and verify with the latest Impex provided. The profile fields imported flawless.

User - Avatar > works but started to hang and emailed me a 2mb email full of human unreadable stuff (the avatar image?). Any pointers what could make the avatatar option work better? I set the avator storage to file system.

User - Homepage > same as before. actual homepage url ads my website as a prefix. (see above posts)

[Jerry]

You have avatars that are over 2meg in size ???

I'll have a look at the size checking so it dosn't attempt to import huge images.

ImpEx grabs the contents of U_Homepage and only that, so I'm not sure what is going on there at all.

[klaus]

- avatar
in UBB avatars are saved as URLs in the U_Pictures field (even URLs to external sites not belonging to my domain) and then they get reseized to my avatars specs on the fly as the avatar image is requested....so one might have a 2meg or otherwise huge file avatar image elsewhere on the internet.....its possible. So seize check would be nice or something that simply skipps on errors and moves on to the next entry.

Also keep in mind avatar are not the biggest issue.

- Homepage
On the homepage.... hmmm.. its strange. UBB and vb3 both show the same fields in the database. the correct URL Example: www.google.com (http://www.google.com/) and if I look at someones profile it displays the propper URL, but when i click on it it ads my own site as a prefix so it actualy becomes http://mysite.com/www.google.com

- Profile
I like how you setup the import BUT one minor detail I noticed...: U_Bio in UBB should import into Biography (field1) instead of field10. Again no big deal as I could remap it in admincp

[klaus]

one more quick update on the homepage issue:

In admincp I just edited a users profile and in the homepage field it shows the URL as www . domain.com only. I changed it to some other domain name and upon saving it changed it to include the http:// in front of it and voila the URL now works without the prefix. So if Impex could 'simply' ad the http:// as a prefix on all URL's then that might solve the problem?

sleep time

[Jerry]

Avatar

I have added a size check to impex now and put that in the ubb_threads module so you can set a limit of the size of avatar to import, note this is the raw byte size and not dimension.

Biography

Have changed that to use the default vBulletin one now opposed to creating a duplicate.

Homepage

URLs are now parsed to ensure that they are prepended with http://

all in version 1.35

[klaus]

I am 99.9% happy with the Impex now and after some more testing will do the real move over the weekend. Two minor issues I just noticed though....

Avatars - Under UBB the avatar was a URL to any image hosted anywhere on the internet OR you may upload an image in the CP. If you upload the image had to match the avatar requirements (file size and pixels) if you simply link to an external image nothing gets checked. So a user could enter the URL to an image they found in googles Image search taht was 1000 pixels wide and 2MB large, then when the forum calls it it simply calls the URL and displays it in the 80x80 or whatever defined space. It reseizes the image and disregards filesize. If the image gets hosted by some other webserver/site they will get hit with x amount of hits. Now when importing I just now imported some of these large images and it become a problem as they get displayed large on top of those users posts and profiles....so the avatar import is not 100% useable and at this point i'll simply ask my users to reset there avatars.

Userfileds - Under UBB there is one field called U_Name, its the real name of a user...if you could include that, nice. if not I'll try to patch 004.php myself.

[Jerry]

Avatars - Under UBB .......

There was a lot of talk about what to do about that as with remote hosting you don't have the controll over the image size and rez thogh you can't limit the display arera with the HTML as you said. Also what can be done has to balanced with what is reasonable with an import as if there were 10,000 users and you were downloading/measuring/resizing every avatar that would cause a huge amount of issues.

Limiting on size and then having an admin/moderator intervien after the import for users that had the big images seemed the best way, as there is always some clean up after an import.



Userfileds - Under UBB there is one field called U_Name....

Done.