multiple language packs. is utf8 sufficient?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Luciano
    Member
    • Apr 2001
    • 87
    • 3.8.x

    multiple language packs. is utf8 sufficient?

    Hello everybody
    I ran into a problem and would need some help

    I want to install vb for a new site.. using multiple languages.
    Unfortunately i have to be on a windows server (different reasons)
    using mysql and apache 2.2

    I want to install following language packs:
    English (master) .. german, french, spanish, russian and chinese

    I set to utf8 in the config file before installing vb (on my localhost)
    then I save the language xml in utf8
    then i try to install vbulletin.. everything is ok.. until importing the phrases..
    then i get a mysql error:

    MySQL Error : Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation ' IN '
    Error Number : 1267
    _________

    If I do not change anything.. do a normal install.. in english and import the chinese xml for examlpe (found here... not as master language) then it works fine.. (i get the correct chinese characters displayed on the board)

    but.. in the language selector.. AND in ALL other dropdowns i only get weird characters !!!!


    here is my question:
    do I only set the config.php and save language.xmls in utf8 before installing?

    or do I have to do something else? on apache or/and mysql or/and php?

    help would be very apreciated, thank you

    Luciano
    Last edited by Luciano; Thu 8 Feb '07, 1:01am.
  • Andy Huang
    Senior Member
    • Feb 2004
    • 4602

    #2
    Colation issue. Since it is new, its good to resolve it now instead of use something odd and witty to cause problems later.
    To fix the error mentioned above on a NEW database, use phpMyAdmin and simply change the collation of all your text / varchar fields to utf8_general_ci and you'll be fine.

    To fix the error mentioned above on an EXISTING database, back up your data first. Open the backup .sql and change DEFAULT CHARSET of each table to:
    Code:
    DEFAULT CHARSET=utf8;
    Create the table again using the collation utf8_general_ci and then restore. Make sure to save the .sql's in UTF-8 w/o BOM.
    Best Regards,
    Andy Huang

    Comment

    • Luciano
      Member
      • Apr 2001
      • 87
      • 3.8.x

      #3
      thank you for your reply..

      Actually i had to reconfigure phpmyadmin to be able to do that..
      and... when I looked.. it was all done...
      strange..
      because i did a couple of installs while testing.. actually i did a query: "ALTER DATABASE vbulletinnew DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;"

      I then emptied the database en did a fresh install.. and all seems working (i had du modify the encoding of the language.xmls to utf8 and also set the encoding in the language settings to utf8)

      everything seems working now!! Except ..little problem with the chinese.. where there are still some characters not displayed.. but they are in the middle of the text..
      could it be a language.xml issue?
      example: over the foum jump dropdown it says
      in english : Forum Jump
      in chinese: 论�?�跳转
      instead of :
      论坛跳转


      or is it just my browser not displaying all signs?

      thankyou for your help

      Luc
      Last edited by Luciano; Thu 8 Feb '07, 12:30pm.

      Comment

      • Andy Huang
        Senior Member
        • Feb 2004
        • 4602

        #4
        Sorry about the late reply. I've been busy all day working.

        Double check if the file looks okay with a text editor that supports unicode utf-8 decoding, as well as chinese characters. I personally use EmEditor, but there are certainly others out there.

        If it looks okay, try to rebuild the language to see if that resolves problem. Admin CP > Language Manager > [ Rebuild All Languages ].

        If you still suspect the file not working, attached is a copy of simplified chinese xml that is offered with the Chinese version of vBulletin, which you can give a try with.

        Other than that, I don't know for sure of the cause either... I'll ask the encoding dood about this issue later when he gets online. Please post back if the issue is still not resolved; I've got this thread subscribed so I'll know when you reply.
        Attached Files
        Best Regards,
        Andy Huang

        Comment

        • Luciano
          Member
          • Apr 2001
          • 87
          • 3.8.x

          #5
          thank you very much for the file..
          the reply was not late.. because of the time shift.. as i am in europe..

          i compared the 2 files mine and yours.. and there are only minimal differences.. the main being the header.. one is master.. mine was not.

          I installed it anyway and.. it seems that the problem is on the side of my server..
          At first the output was questionmarks only..
          after playing around a little bit (changing the first line from utf8 to ?xml version="1.0" encoding="ISO-8859-1"?) - no special reason the base encoding of the file remaining utf8 ..
          it started behaving like my old file...

          example: the field: 1 hour ago
          in my texteditor (I use edit plus) it says:
          <phrase name="1_hour_ago" date="0" username="" version=""><![CDATA[一小时前]]></phrase>
          which seems correct for me..
          when i go edit the phrases i see:
          一��?时��?
          so i replace the line with:
          一小时前
          and save...
          output is then again:
          一��?时��?

          i enabled oriental character display on in windows (thinking this could be the issue on the server side) but it was allready enabled..

          it looks to me as if I have german special characters encoded in utf-8 and trying to display them in a ISO-8859-1 enabled browser. i then also get for ä or ü only a , the other characters being ok.

          writing these lines it just occurs to me i DID change the first line from utf to iso in order to be able to see anything else than ???? (questionmarks) .. so I will look into that.
          but if i leave the first line:
          <?xml version="1.0" encoding="UTF-8"?>
          i ONLY get questionmarks as output after import.


          I have to go to work now.. but I will play around with this when i get back in a few hours.. and let you know..

          thank you for your support and your time

          Luc
          Last edited by Luciano; Thu 8 Feb '07, 8:14pm.

          Comment

          • Luciano
            Member
            • Apr 2001
            • 87
            • 3.8.x

            #6
            Problem solved !!!! And a big THANK YOU to Andy

            Might help others:

            1. I did everything you said.. nothing helped.. i think i reinstalled vbulletin 20 times.. But i kept the conversions of the database to UTF-8 ...
            still got an error: Illegal mix of collations .. in the admin help popups!!!
            because there is a FIND_IN_SET in a query that mixes varchar and int fields or something like that.. anyway it didnt work



            2. I modified apache conf and mysql (my.ini) by adding
            in MYSQL my.ini twice in the client part and the serverpart:

            [client]
            default-character-set=utf8

            [mysqld]
            default-character-set=utf8

            and in the apache conf. file i added:

            AddDefaultCharset UTF-8

            .... but still no way.. i got the missing characters with the file from here.. and the question marks in andys file..
            ------------
            3. So i did.. what i should have done in first place.. I read the instructions!!!
            in the config.php of vbulletin it says:
            Try to use mysqli if you are using PHP 5 and MySQL 4.1+
            so i installed the php module for mysqli (had never heard of it before)

            and activated the mysqli option in the vbulletin config.php
            i also activated the other 2 options:
            $config['Mysqli']['charset'] = 'utf8';
            and
            $config['Mysqli']['ini_file'] = 'C:\apache\mysql\my.ini';

            and voila.. i had the file with the questionmarks loaded.. and all signs were there.. !!!! perfect..

            I actually i dunno if all steps are needed.. but.. I think 1. in any case.. the conversions are important.. 2. i dunno if its really necessary to change the config files.. of apache and mysql .. as for 3. it wont work without it.. at least not on my system.

            Thanks again for your help

            Luciano
            Last edited by Luciano; Fri 9 Feb '07, 3:13am.

            Comment

            • Andy Huang
              Senior Member
              • Feb 2004
              • 4602

              #7
              My apologies, I was just told that there's a slight modification done on the Chinese ver to made that work better... Give this a try, I don't know if this is the magical fix, but it might fix the problem:

              Open up includes/class_xml.php, find:
              PHP Code:
              function &parse($encoding 'ISO-8859-1'$emptydata true
              Replace that with:
              PHP Code:
              function &parse($encoding 'UTF-8'$emptydata true
              Save, upload, overwrite, and try to import your language again to see if that resolves the issue for you.



              Edit:
              Ah, glad to see it worked out for you
              Best Regards,
              Andy Huang

              Comment

              • Luciano
                Member
                • Apr 2001
                • 87
                • 3.8.x

                #8
                Thank you for the fix!

                2 issues..

                i made a test post in chinese without installing it the fix..
                in the post.. i still had some characters not displaying like in the first example..
                the phrases though were ok!

                i then installed you fix.. i also changed the first line of the xml from
                <?xml version="1.0" encoding="ISO-8859-1"?>
                to
                <?xml version="1.0" encoding="UTF-8"?>
                IT did not work!!!

                then i went back to iso (first line in xml) and it worked!!!

                also i did a second post in my testforum. copy and pasting some phrases into the post.. and it worked !!!

                BUT when i copy and pasted the same phrase into the first post (the one with missing chars.. i still have the same errors)

                So what i will do now.. is simply delete that first post and use it as it is.. With that modification in the parser function.

                Never touch a running system!!!

                Luc

                But i still dont unterstand why it doesn't work with the first line set to:
                <?xml version="1.0" encoding="UTF-8"?>
                maybe because i installed the english language first? and it has that first line??
                EDIT: well in this version of vb i downloaded from here.. in the install files i did a search for ISO-8859-1 and it seems that the install files set everything back to iso.

                AND I think the problems come from the fact that the locale of the machine is not chinese...
                Last edited by Luciano; Fri 9 Feb '07, 7:26pm.

                Comment

                Related Topics

                Collapse

                Working...