Able to cont Impex

[lasto]

i have tried various ways of moving my attachments to new board and keep getting errors like this :

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 9492225 bytes) in /admincp/impex/ImpExDatabaseCore.php on line 1306

i doing 10 attachments at a time and taking them from the file system and then putting them into the mysql database.


Just updating the ImpExDatabaseCore.php from new version of impex

and still it happens

Importing 10
From : 700 :: To : 710


Failed :: Attachment not imported
Failed :: Attachment not imported
Failed :: Attachment not imported
Failed :: Attachment not imported
66.67% Attachment -> dead 2.jpg
66.67% Attachment -> dead meat.rar
66.67% Attachment -> desp 2.jpg
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 8460365 bytes) in /admincp/impex/ImpExDatabaseCore.php on line 1306

[StrongMotive]

Your PHP does not have enough memory allocated to it to complete this operation. Ask your host to increase the 'memory_limit' variable in php.ini to at least 32M.

To temporarily up your limits edit your includes/config.php file and add these lines right under the <?php line:

ini_set('memory_limit', 32 * 1024 * 1024);

[lasto]

hi m8 cheers for your help because if cant sort out this attachment problem i have to restart the board again and move attachments for well over 4000 threads.Not a task i look forward to.

Will send email to host now.

[lasto]

i have over 33 gigs of attachments will these store in the sql database as im impexing them there from my old board to new board so i can start afresh.Just dont want to start it and then find out mysql cant handle this amount of imformation.

[lasto]

because it crashed out on attachments had to restart it all again - and now when i finally get round to doing the attachments again its just a blank screen - nothing changes at all.(Also ive just grabbed latest impex from here and uploaded that and its still same)

[lasto]

3rd try now and this is all that happens when i go to move attachments - correct path as well - its not throwing up no errors its just not doing anything


Remove ImpEx once import is complete and final
ImpEx build version : 1.85

Database cleanup & restart ||| Online manual
Using ImpExConfig.php for target config.

[lasto]

i have managed to impex everything from my old forum such as threads/posts/polls avatars etc but when i go to attachments the screen goes blank and thats it.
How do i get it to work without having to import all the information again as selecting clear ids or the sessions removes everything so i have to import all over again just so i can get to the attachments option.

[lasto]

Your PHP does not have enough memory allocated to it to complete this operation. Ask your host to increase the 'memory_limit' variable in php.ini to at least 32M.

To temporarily up your limits edit your includes/config.php file and add these lines right under the <?php line:

ini_set('memory_limit', 32 * 1024 * 1024);


added the line to config.php and it worked for a while then crashed out again.


is 32 the max u are allowed to allocate.






Importing 20
From : 680 :: To : 700


66.67% Attachment -> powers series.jpg
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2538179 bytes) in /admincp/impex/ImpExFunction.php on line 180




changed to 128 in config file and still got same error

Importing 20
From : 680 :: To : 700


Failed :: Attachment not imported
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2538179 bytes) in /home/custom/public_html/ccnew/admincp/impex/ImpExFunction.php on line 180

[StrongMotive]

32 is not the maximum you can go up to 64, but the server are using may suspend you for overused resource - if you have your own server then its fine to do it while no one else is browsing (as it will be slow for them)

[lasto]

im on a dedicated so im fine but i even changed it to 512 and still got same error as listed above.

So looks like im out of solutions.

as you can see here i have it set to 24

PHP 5.2.4 PHP
Max Post Size 8.00 MB
PHP Maximum Upload Size 2.00 MB
PHP Memory Limit 24.00 MB
MySQL Version 4.1.22-standard
MySQL Packet Size 8.00 MB


some of the attachments on my board i know are about 8 meg in size as we set the board up so members can upload upto a size of 8megs.

[Steve Machol]

Does impex show if the attachments imported?

[lasto]

yeah ive had it where its done a few but most of the times it just goes blank.
Once it does that the only way is erase the database and impex everything back in again because deleting session ids does not allow you to go straight back and carry on trying to import attachments - you need to start at top with checking databases and work your way back down the list.

Very annoying.

[Steve Machol]

Can you please post a picture of the completed modules? I still don't understand.

[lasto]

i cant post a pic as i have to delete all session ids in impex to go back to main screen.
Everything is impexed over except the attachments - it starts it and then comes up with an error like this :

Importing 20
From : 680 :: To : 700


Failed :: Attachment not imported
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2538179 bytes) in admincp/impex/ImpExFunction.php on line 180

I have increased the phpini memory limit as you can see by the stats below but same error all time.

Server Type Linux
Web Server Apache v1.3.39
PHP 5.2.4
PHP Max Post Size 8.00 MB
PHP Maximum Upload Size 2.00 MB
PHP Memory Limit 32.00 MB
MySQL Version 4.1.22-standard
MySQL Packet Size 8.00 MB

[Steve Machol]

Your PHP does not have enough memory allocated to it to complete this operation. To temporarily up your limits edit your includes/config.php file and add these lines right under the <?php line:

ini_set('memory_limit', 128 * 1024 * 1024);

[lasto]

sorry steve ive already done that today.

here is the stats after i edited config file
PHP Memory Limit 128.00 MB

so i try again - Failed :: Attachment not imported
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2538179 bytes) in /impex/ImpExFunction.php on line 180

so as you can see increasing the PHP memory limit has no effect on it at all.

[Steve Machol]

This means that your server is not recognizzing this change. I'm afraid you are going to need to ask your host to change this directly in php.ini, then restart Apache.

[lasto]

im on a dedicated i can do it myself.
It is recognized because it says so in admincp stats whihc i posted above

i will restart apache now and see what happens

PHP Memory Limit 128.00 MB

[lasto]

kk here goes i changed a few setting as u can see

PHP 5.2.4
PHP Max Post Size 50.00 MB
PHP Maximum Upload Size 50.00 MB
PHP Memory Limit 256.00 MB
MySQL Version 4.1.22-standard
MySQL Packet Size 8.00 MB

and restarted apache and still same error - how much memory does this program actually want :)


Failed :: Attachment not imported
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2538179 bytes) in admincp/impex/ImpExFunction.php on line 180

[Steve Machol]

This error clealy shows that your server is not uing 128M:

Fatal error: Allowed memory size of 33554432 bytes exhausted

Also are you still using Impex instead of upgrading as we recommended?

[lasto]

i cant upgrade because for the simple reason after upgrading 3.5.4 to 3.6.8 the board has quite a few bugs so am getting everything of use to a new board on the same server so i can kill off the old one.

also upgrading would not remove any hacks that are in the database would it so why would i want to upgrade a board that has hacks from a few yrs back.

Impex does other boards so why not its own :)

[Steve Machol]

You have a 33GB attachment table - that's why. You are going to need tons of memory and time. And even your last change did not take hold because your server is still only allowing 32M.

I do not know how to make this work. The recommended method is to upgrade. Impex was not designed for this. Maybe Jerry will have some further thoughts, but in the meantime you need to investigate why your server refuses to allocate more memory.

[Jerry]

I've run PHP at 512 Meg for large imports, the attachment is being read it memory to be written to the vBulletin file store or database.

The only thing is either lower the per page even more or raise the memory, or do it on another server, move the attachments to the database then move the database and restore it.

[Jerry]

Threads merged.

33 gig :eek: ........ *thinks* :cool:

[lasto]

so basically what u saying is upgrade my old board to 3.6.8 as it be a lot easier but you must admit if i do that it will still leave a lot of crap in the database which is redundant and no longer needed from old hacks.

Such as extra tables from hacks that have been previously installed so it wont be a fresh board as any of these tables can throw up errors.

[Wayne Luke]

Such as extra tables from hacks that have been previously installed so it wont be a fresh board as any of these tables can throw up errors.

If a table is not being accessed anymore then it shouldn't be able to throw errors. However you can compare your installation with the tables in installs/mysql-schema.php for 3.6.8 and delete any extraneous fields or tables from the database. That is actually the preferred method. ImpEx is not an upgrade tool nor is it "database cleaning" tool.

[Steve Machol]

You can delete any extra tables from hacks (assuming these are not default vB tables of course.) That hardly requires any effort at all.

[lasto]

ive decided to scrap this idea and just import the attachments to the threads manually.
Is over 4000 threads to do so will take some time doing them but at least i know the attachments will get there fine.

[lasto]

well heres hoping - left it running overnight last night and it still doing the attachments but must admit its very slow.

Total Attachments 2,952
Disk Space Used 2.64 GB


is what it has moved so far - there is 26,000 of them in total so may take some time unless it crashes out while im at work today.

[lasto]

well from this morning to now stats aint changed much - at this rate it will take about another week to do the attachments.

Total Attachments 3,246
Disk Space Used 2.80 GB

and here it is again

Importing 2
From : 4100 :: To : 4102


66.67% Attachment -> EFNY Preview.jpg
66.67% Attachment -> Escape From New York.rar

[lasto]

nearly 24 hrs later

Total Attachments 5,735
Disk Space Used 6.19 GB

its got to go to 33 gig :(

[lasto]

and its still going on LOL

From : 12172 :: To : 12174


66.67% Attachment -> Fearless.rar
66.67% Attachment -> Shrek3.rar

[Jerry]

There isn't much "LOL" about it, I'm quite happy that it's still running, proves the stability if anything !

Taking out the hacks you had put in would of been a lot quicker and the preferred method opposed to using ImpEx to clean it up as you are going to loose all kinds of other data that ImpEx doesn't import as well.

[lasto]

Total Attachments 19,119
Disk Space Used 23.29 GB


still going - gonna stick with this and see how long it takes to move 30 gigs of attachments set at 2 at a time.

what other data am i going to lose ?

Thread views etc not to bothered about them

EDIT

Total Attachments 19,937
Disk Space Used 24.38 GB

[lasto]

getting there slowly :)

Total Attachments 23,629
Disk Space Used 29.61 GB

[Griffin80]

lol omg man, let us know how it goes. a week and a half :S