Search issue trying to search in Arabic language

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • ienajah.com
    Member
    • May 2018
    • 92
    • 5.7.5

    [Bug / Issue] Search issue trying to search in Arabic language

    Search function in not working correctly when searching Arabic language the search word comes in strange form and not giving results please see below
    search word
    Click image for larger version  Name:	AS.png Views:	1 Size:	34.3 KB ID:	4391461

    search result

    Click image for larger version  Name:	AR.png Views:	1 Size:	52.7 KB ID:	4391462
    search result choosing the forum Arabic language

    Click image for larger version  Name:	AR2.png Views:	1 Size:	58.2 KB ID:	4391463


    however when trying to search English word it works just fine see below

    search work

    Click image for larger version  Name:	ES.png Views:	1 Size:	35.5 KB ID:	4391464

    search result

    Click image for larger version  Name:	ER.png Views:	1 Size:	99.2 KB ID:	4391465



























    note that : I rebuilt the search index many times and I'm getting same results

    the links for Arabic and English nods look friendly and fine in the browser see below

    Click image for larger version  Name:	lin.png Views:	1 Size:	109.4 KB ID:	4391466

    but when I copy / past the link it comes with strange order

    Code:
    https://www.ienajah.com/vb/forum/%D9%83%D9%80%D9%80%D9%84%D9%80%D9%80%D9%8A%D8%A9-%D8%A7%D9%84%D9%80%D9%87%D9%80%D9%86%D8%AF%D8%B3%D9%80%D9%80%D8%A9-%D8%A7%D9%84%D9%80%D8%B5%D9%80%D9%86%D9%80%D8%A7%D8%B9%D9%80%D9%8A%D9%80%D8%A9/%D8%A7%D9%84%D9%80%D9%85%D9%80%D9%83%D9%80%D8%AA%D9%80%D8%A8%D9%80%D8%A9-%D8%A7%D9%84%D8%A5%D9%84%D9%80%D9%83%D9%80%D8%AA%D9%80%D8%B1%D9%88%D9%86%D9%80%D9%8A%D9%80%D8%A9/104326-%D9%85%D8%A7%D8%AF%D8%A9-%D9%85%D8%B3%D8%A7%D9%82-%D9%85%D9%82%D8%AF%D9%85%D8%A9-%D9%81%D9%8A-%D8%A7%D9%84%D9%87%D9%86%D8%AF%D8%B3%D8%A9-%D8%A7%D9%84%D8%B5%D9%86%D8%A7%D8%B9%D9%8A%D8%A9-introduction-to-industrial-engineering
  • Wayne Luke
    vBulletin Technical Support Lead
    • Aug 2000
    • 73981

    #2
    It appears that you are not using a UTF-8 database character set and collation. We recommend using the UTF8MB4 character set and utf8mb4_general_ci collation in MySQL for this reason. You would have to convert your database.

    The link is normal since you can't actually use UTF-8 in a URL. We have to convert it to its encoded HTML entities so the browser can process it correctly.
    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud demonstration site.
    vBulletin 5 API

    Comment

    • ienajah.com
      Member
      • May 2018
      • 92
      • 5.7.5

      #3
      yes the data base is mixed now between latin1 and UTF8 characters.

      since I carried out the update from 3.8.0 directly to 5.4.1 with enabling the the Latin1 from the config file.

      could you please guide me in changing it or provide me with the method.

      since I checked alot of methods and non of them gave good results.

      Comment

      • ienajah.com
        Member
        • May 2018
        • 92
        • 5.7.5

        #4
        Screenshot of data base
        Attached Files

        Comment

        • Wayne Luke
          vBulletin Technical Support Lead
          • Aug 2000
          • 73981

          #5
          The option in the /core/includes/config.php file has no control over how the tables are created. It only controls the language that vBulletin talks to MySQL in.

          When creating tables, vBulletin uses the character set and collation defined as the default for the database. This is done in MySQL. As long as this is properly set, then all tables and the fields in them will have the same character set and collation. Since you have a mixture, it seems that somewhere along the line the defaults for the database were changed. Again we have no control over this in an upgrade. If we create a vBulletin 5 database during the installation process, it has a UTF8MB4 character set and the utf8mb4_general_ci collation.

          Here are the instructions on changing your database's character set and collation:


          Here is more information:


          For the best results, you should do this on a backup copy of your database. Better yet, restore your 3.8.X database, make the changes and set your defaults. Then upgrade to vBulletin 5 again.
          Translations provided by Google.

          Wayne Luke
          The Rabid Badger - a vBulletin Cloud demonstration site.
          vBulletin 5 API

          Comment

          • ienajah.com
            Member
            • May 2018
            • 92
            • 5.7.5

            #6
            thanks Wayne your link was really helped me to understand this ...

            On data base copy I followed the instructions and changed the tables on e by one / data base to UTF8MB4 character set and the utf8mb4_general_ci collation, 3 tables where having error (tag, phrase, userchangelog) see below (could you guild me in solving them)

            ALTER TABLE tag CONVERT TO CHARACTER SET UTF8MB4 COLLATE utf8mb4_general_ci
            MySQL said: Documentation
            #1062 - Duplicate entry 'ÇáäÌã' for key 'tagtext'

            ALTER TABLE phrase CONVERT TO CHARACTER SET UTF8MB4 COLLATE utf8mb4_general_ci
            MySQL said: Documentation
            #1062 - Duplicate entry 'common.langDirLtr-2' for key 'name_lang_type'

            ALTER TABLE userchangelog CONVERT TO CHARACTER SET UTF8MB4 COLLATE utf8mb4_general_ci
            MySQL said: Documentation
            #1071 - Specified key was too long; max key length is 1000 bytes
            Click image for larger version

Name:	ddada.png
Views:	373
Size:	190.3 KB
ID:	4391532
























            then try to connect the database copy and keeping core/includes/config.php to read from latin1 it is connecting but still having the same issue of search and URL format cause table values seams to be still in Latin1 / latin1_swedish_ci, I could not find direct query to change the table value could you provide me with it please.

            Click image for larger version

Name:	latin.png
Views:	311
Size:	100.4 KB
ID:	4391533




            Comment

            • Wayne Luke
              vBulletin Technical Support Lead
              • Aug 2000
              • 73981

              #7
              When you set your language to UTF-8 in vBulletin under Languages & Phrases -> Phrase Manager, does the content appear correctly on the forums?
              Translations provided by Google.

              Wayne Luke
              The Rabid Badger - a vBulletin Cloud demonstration site.
              vBulletin 5 API

              Comment

              • ienajah.com
                Member
                • May 2018
                • 92
                • 5.7.5

                #8
                first I'm working on the current data base mixed between latin1 and UTF8 ..... not on the copy mentioned earlier


                if I understand correctly you mean in vBulletin under Languages & Phrases -> Languages Manager-> [Edit Settings] -> HTML Character Set

                cause I could find in phase manager any place to change to UTF

                (I have two languages installed Arabic and English in english )

                Click image for larger version

Name:	Screen Shot 2018-05-16 at 9.10.36 PM.png
Views:	345
Size:	912.2 KB
ID:	4391581




























                I used WINDOWS-1256 when I change it to UTF8 their most of the content appeared in strange order and some fine !!!!!!! and search word appeared OK !!!!!!!!, still no results (this in English language )

                Click image for larger version

Name:	Screen Shot 2018-05-16 at 9.11.53 PM.png
Views:	294
Size:	1.19 MB
ID:	4391582



























                inClick image for larger version

Name:	2121212.png
Views:	298
Size:	1.71 MB
ID:	4391583 threads































                however in arabic language it is fine and every thing looks normal every where except in Admincp strange order appeared , search issue still their and no result appears and also strange order

                Click image for larger version

Name:	2222222.png
Views:	299
Size:	864.6 KB
ID:	4391584
































                it seams I have an Issue with the language file ??????????? please advice

                Comment

                • Wayne Luke
                  vBulletin Technical Support Lead
                  • Aug 2000
                  • 73981

                  #9
                  We will need a copy of your database to determine where the issue is. You can make a copy available by uploading it to the Cloud Sharing service of your choice and opening a support ticket by email [email protected] from the email address on your customer account. You should convert back to your previous version until this can be resolved.
                  Translations provided by Google.

                  Wayne Luke
                  The Rabid Badger - a vBulletin Cloud demonstration site.
                  vBulletin 5 API

                  Comment

                  • ienajah.com
                    Member
                    • May 2018
                    • 92
                    • 5.7.5

                    #10
                    email Already sent and support ticket open notification received

                    Comment

                    • cashu
                      Senior Member
                      • Mar 2007
                      • 107

                      #11
                      i have the same issue please anyone here tell me how we can fix it ?

                      Comment

                      • Wayne Luke
                        vBulletin Technical Support Lead
                        • Aug 2000
                        • 73981

                        #12
                        Originally posted by cashu
                        i have the same issue please anyone here tell me how we can fix it ?
                        Please rebuild your search engine.
                        Translations provided by Google.

                        Wayne Luke
                        The Rabid Badger - a vBulletin Cloud demonstration site.
                        vBulletin 5 API

                        Comment

                        • cashu
                          Senior Member
                          • Mar 2007
                          • 107

                          #13
                          i do that but dosnt fix it

                          Comment

                          Related Topics

                          Collapse

                          Working...