Fixing invalid characters?

[funkmiester]

After my import I see a lot of thread titles which have invalid characters for example quotes "" show as ' - I also have foreign languague characters showing as their ascii equivalents.

I've manually edited them to be correct and in the message when viewing it, it shows the corrected version. But when looking at the forum listings, it still shows the imported version with the invalid characters.

How can I update the forum listing titles? I chose "Rebuild Thread" and "Rebuild Forum" information, but it hasn't made a difference. Is their some other method to make these changes permanent in the forum listings?

Thanks.

[Jerry]

I take it you are editeding the posts ? Then the error in the text is still there in the thread listsing ?

I think what would be happening is that its the text in "thread.title" that is stored with the thread table and not in the "post" table that is the issue.

If this is the case I think me extending tools/cleaner.php to also clean thread titles would be the best thing to do.

Can you check thread.title in the database to see if that is it?

If it is I'll update cleaner.php (though I think I'll do that anyway as it should do titles to be honest).

[funkmiester]

Yes, I was editing the post titles, which were correctly modified when looking at the post itself. But it's those same titles in the forum listings that aren't changing.

So, yes your theory is right, see attachment of the database entry.

So, there is a "cleaner" I can run to fix this and other things? Is this something you can send me?

Let me know, thanks.

[Jerry]

The cleaner is in the tools dir of the impex download so you should have it, though I'll need to add the thread.title to it because it dosn't cover that at the moment.

[Jerry]

Ok, i've updated impex/tools/cleaner.php now to do the thread titles as well.

And updated the download so you can grab it now.

[funkmiester]

Ok, thanks Jerry - I'll give it a whirl and let you know the outcome.

[funkmiester]

Jerry - I ran "cleaner.php" on my local system which is a backup of my live system and the problem still persists unfortunately. The thread titles remain unchanged. I did see at the bottom of the resulting cleaner screen that it had done the relevant threads, but I see no change.

After seeing no difference, I ran the relevant "Update Counters", but still no change.

Thoughts?

[Jerry]

"Update Counters"

Does exactly that, it won't change the contents of a text field.

What is the exact srtring as an example, and what did you put in the array in cleaner.php ?

[funkmiester]

Ok, duh on me :p

I just ran it as is, not realizing there were options I needed to insert!

So do I need to find all these anomalies and put them in the array? Can I run this safely multiple times (in case I miss some on the first try)?

Aside from the array, what other sections in cleaner.php do I need to edit, if any? Sorry, this is my first time with it, so don't have much of a clue :D

[Jerry]

Ok, duh on me :p

I just ran it as is, not realizing there were options I needed to insert!

Well I do my best, though it does need to know what its looking for ;)


So do I need to find all these anomalies and put them in the array? Can I run this safely multiple times (in case I miss some on the first try)?

Yes you can run its lots of times as you find things to clean out, obviously doing more that one per pass is a lot faster.



Aside from the array, what other sections in cleaner.php do I need to edit, if any? Sorry, this is my first time with it, so don't have much of a clue :D

Just the array. Left hand side what you want it to find, right hand side what you want it to be.

There is these boolean switches :



$do_posts = true;
$do_sigs = true;
$do_thread_titles = true;


Depending on what you want to clean as well.

[funkmiester]

Thanks Jerry, worked perfectly and fixed not just the titles but also messages with invalid characters :cool:

I just have a couple of other issues outstanding that you had started to help me with a few days ago, you'll find them here (http://www.vbulletin.com/forum/showthread.php?p=761390#post761390).