PDA

View Full Version : After Large Import



LoTGoD
Wed 21st Jan '09, 3:08pm
I followed the instructions here for the med/large board imports :

http://www.vbulletin.com/docs/html/med_large_import

My import has finished, but I am a bit confused as to what I should do next...

Specifically, the first part here

Manually dropping the fulltext-index 'title' (title, 'pagetext') from the post table before the post module, then adding it after the import has been run can speed up the post module considerably.
ALTER TABLE `post` DROP INDEX `title`;
ALTER TABLE `post` ADD FULLTEXT `title` (`title` ,`pagetext`);Should I have ran both SQL queries before I got to the post module, or just the first one, and then ran the second query after the import finished?

Also, I did set:
define('shortoutput', true); but I didn't see a difference in the text that was displayed during the import.

Per a post I saw from Jerry, I set my memory limit to 1024mb for MySQL, and I think I was able to turn off time outs, but I would like to know specifically how you turn the timeout off. (This is a local server that I have setup and control over)

Once the import is completed, should I run some SQL query to remove these? :

ALTER TABLE `post` ADD INDEX `idx_importpostthread` ( `importpostid` , `importthreadid` )
ALTER TABLE `thread` ADD INDEX `idx_importthread` ( `importthreadid` ) What would the SQL code be to remove those?

Thanks for any help on this issues.

Jerry
Thu 22nd Jan '09, 2:32pm
When doing the large imports for speed, you do this before the import :



ALTER TABLE `post` DROP INDEX `title`;


Then this after :



ALTER TABLE `post` ADD FULLTEXT `title` (`title` ,`pagetext`);


So the building of the fulltext index dosen't slow it down during the import, and it's quickier to bulk build it afterwards.

This :



define('shortoutput', true);


Effects some of the display returned by impex opposed to the data of the import.


but I would like to know specifically how you turn the timeout off.

In the php.ini file :

http://www.php.net/manual/en/info.configuration.php#ini.max-execution-time




Once the import is completed, should I run some SQL query to remove these? :


You can do, it would be considered good house keeping in so much as "if it isn't explicitly needed, clean it out".

If you do any consecutive imports it will be re-created.

LoTGoD
Thu 22nd Jan '09, 3:37pm
Thanks for the response Jerry. May I suggest modifying the Impex Support section or the vBulletin manual for Med/Large Imports to better explain the procedure for the Full Text modification? I don't think I will be the only one that is confused as to when it is advised to run those queries.

To clean out the import code, would I run this?


ALTER TABLE `post` DROP INDEX `idx_importpostthread` ( `importpostid` , `importthreadid` )


ALTER TABLE `thread` DROP INDEX `idx_importthread` ( `importthreadid` )

Jerry
Thu 22nd Jan '09, 4:02pm
Thanks for the response Jerry. May I suggest modifying the Impex Support section or the vBulletin manual for Med/Large Imports to better explain the procedure for the Full Text modification? I don't think I will be the only one that is confused as to when it is advised to run those queries.

Agreed, the pressure is very much mounting to re-write/update the manual ;)


To clean out the import code, would I run this?

Yes :)