Yahoogroup Import

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • amphicar770
    Member
    • Mar 2005
    • 37
    • 3.0.0 Release Candidate 4

    Yahoogroup Import

    I've gotten all of the posts from the Yahoo Groups into a MS Access database using PG Offline... Jerry converted these to MySQL (A++ for customer support!).

    I am now attempting import. On the check and update database step all goes well with following two exceptions ...

    Possibly custom tables or incorrect prefix :

    Groups NOT found.
    System NOT found.

    Do I need to fix this before proceeding? If so, how?

    Thanks
  • Jerry
    Senior Member
    • Dec 2002
    • 9137
    • 1.1.x

    #2
    In ImpExConfig.php ensure you have this :

    PHP Code:
    $impexconfig['source']['tableprefix']   = 'tbl'
    I wrote ImpEx.

    Blog | Me

    Comment

    • amphicar770
      Member
      • Mar 2005
      • 37
      • 3.0.0 Release Candidate 4

      #3
      Just updated it. Still same error (full listing below).

      ImpEx build version : 1.87

      Database cleanup & restart ||| Online manual
      Using includes/config.php for target config.


      Altering tables

      ImpEx will now Alter the tables in the vB database to include import id numbers.
      This is needed during the import process for maintaining refrences between the tables during an import.If you have large tables (i.e. lots of posts) this can take some time.

      They will also be left after the import if you need to link back to the origional vB userid.

      moderator - importmoderatorid OK
      usergroup - importusergroupid OK
      ranks - importrankid OK
      poll - importpollid OK
      forum - importforumid OK
      forum - importcategoryid OK
      user - importuserid OK
      style - importstyleid OK
      thread - importthreadid OK
      post - importthreadid OK
      thread - importforumid OK
      smilie - importsmilieid OK
      pmtext - importpmid OK
      avatar - importavatarid OK
      customavatar - importcustomavatarid OK
      customprofilepic - importcustomprofilepicid OK
      post - importpostid OK
      attachment - importattachmentid OK
      pm - importpmid OK
      usernote - importusernoteid OK
      phrase - importphraseid OK
      subscription - importsubscriptionid OK
      subscriptionlog - importsubscriptionlogid OK

      Valid found tables :


      Possibly custom tables or incorrect prefix :

      Groups NOT found.
      System NOT found.

      If you have all red tables, i.e. none correct this could possible be your table prefix :


      tbl Failed Check database permissions and connection, or table prefix to ensure its correct
      =========================
      I also did try database cleanup and restart but get the following
      Impex Help page

      ImpEx can not read the target database details from impex/ImpExConfig.php OR ../includes/config.php.
      Please enter the target database details in ImpExConfig, or run ImpEx installed opposed to standalone.

      Comment

      • Jerry
        Senior Member
        • Dec 2002
        • 9137
        • 1.1.x

        #4
        Then you need to put the correct details into ImpExConfig for the target database, and I'd double check the source as well.
        I wrote ImpEx.

        Blog | Me

        Comment

        • amphicar770
          Member
          • Mar 2005
          • 37
          • 3.0.0 Release Candidate 4

          #5
          Ah, so close. Found the one table. Now showing:

          Valid found tables :

          tblGroups found.

          Possibly custom tables or incorrect prefix :

          tblSystem NOT found.

          Update: In looking at the imported database, there is no such table. Only 3 tables exist. Groups and two Ygr_ tables.
          Last edited by amphicar770; Mon 28 Jan '08, 4:56pm. Reason: Update

          Comment

          • amphicar770
            Member
            • Mar 2005
            • 37
            • 3.0.0 Release Candidate 4

            #6
            OK, I think I've got it.

            I created a junk table called tblSystem with two fields foo and bar. The table doe not appear to be used, Impex just likes knowing it is there.

            I then ran into an SQL error. Some of my tables and the original forumid had a "-" dash character. MySQL apparently does not like this. I changed it to an underscore "_" and all is well.

            Import of messages is now under way. Currently on message 8,000 - 10,000 (it was a busy Yahoogroup).

            Thanks again for all the assistance.

            Comment

            • amphicar770
              Member
              • Mar 2005
              • 37
              • 3.0.0 Release Candidate 4

              #7
              Now I am confused!

              Import completed successfully. Reported 14,608 success with 3,388 fails. I can live with that.

              The problem I know have is that any editing / html features show up as codes. How do I clean this up? For example,

              <table border=0 cellpadding=2 cellspacing="0"><tr><td>
              <pre>Like Tommy says, that "Amphicar Water Horsepower Statement" suitable for
              printing and tossing into the glove compartment can be found at
              http://www.amphicar.com/WaterHorsepowerStatement.htm . This should get you
              into those restricted bodies of water with that commonly encountered 10 hp
              Last edited by amphicar770; Mon 28 Jan '08, 6:05pm.

              Comment

              • Jerry
                Senior Member
                • Dec 2002
                • 9137
                • 1.1.x

                #8
                That is one of the main things of the yahoo import is the amount of HTML in the source, ImpEx does what it can though cleaning all of it out is a big job.

                If you have a lot of commonly repeated HTML you can take it out with cleaner.php.
                I wrote ImpEx.

                Blog | Me

                Comment

                • amphicar770
                  Member
                  • Mar 2005
                  • 37
                  • 3.0.0 Release Candidate 4

                  #9
                  Since this will be an "archive only" form (no new posts) I set the html option to True and that seems to make it happy. I am assuming that since it is "read only" that this does not present any security risk?

                  I did notice that there are a lot of duplicate messages within threads. Looks like if a thread had any replies that I see
                  - Original Msg
                  - Original Msg
                  - Reply
                  - Reply

                  I may try rerunning with dupe_checking set to true. Based on code comments, I was not really clear on if this will work in importing Yahoogroup DB??

                  Thanks.

                  Comment

                  • amphicar770
                    Member
                    • Mar 2005
                    • 37
                    • 3.0.0 Release Candidate 4

                    #10
                    Imported Forum Msgs Are Nested

                    The forum created from my Yahogroup db exhibits odd behavior not found in my other forums. The message replies are all nested. When there are too many nested they push the page out too wide... is there anyway to stop them from nesting?

                    You can see what I mean here (along with the duplicative messages after import that makes the nesting even more annoying)...



                    Thanks in advance.

                    Comment

                    • Onimua
                      Senior Member
                      • Apr 2005
                      • 4572

                      #11
                      Create a new style with no parent, and see if that still happens. If it does, check the posts to see if there HTML in the posts that is parsing (that's if you allow HTML to be posted in the first place).

                      It could also be due to modifications you have installed as well, but first see if you can replicate this with the default style.
                      Congratulations on the death of vBulletin, Internet Brands.

                      Comment

                      • amphicar770
                        Member
                        • Mar 2005
                        • 37
                        • 3.0.0 Release Candidate 4

                        #12
                        Cleanup question

                        Hi All,

                        I moved my imported yahoogroup messages into my site as forum #18.

                        I would like to do some additional cleaning with the cleanup tool. Can I point it at any forum id or does the one I am working on need to be moved to a specific place?

                        Thanks.

                        Comment

                        • amphicar770
                          Member
                          • Mar 2005
                          • 37
                          • 3.0.0 Release Candidate 4

                          #13
                          OK, answered my own question. Looks like it cleans up everything in the database. Works well.

                          Comment

                          • amphicar770
                            Member
                            • Mar 2005
                            • 37
                            • 3.0.0 Release Candidate 4

                            #14
                            thanks. Running cleaner seems to have fixed the nesting issue. I still have lots of duplicate posts within threads. Seems like most messages are there twice?

                            Comment

                            • amphicar770
                              Member
                              • Mar 2005
                              • 37
                              • 3.0.0 Release Candidate 4

                              #15
                              Duplicate mshs in threads fro Yahoogroup

                              Hi All,

                              Looks like my import from Yahoogroups is nearly complete. One BIG problem remains.

                              It seems like a very large number of posts within a thread are repeated twice.
                              It seems like a very large number of posts within a thread are
                              repeated twice.

                              Any ideas on how to eliminate the duplicates?

                              Thanks.

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...