After upgrade to vBulletin 5, Database size very large

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Oracle simsim
    Senior Member
    • Jul 2009
    • 117

    After upgrade to vBulletin 5, Database size very large

    Hello
    I have large forum ( 3m posts, 380k thread, 120k blog entries, 60k member, 1m visitor messages)

    Database size before upgrade was 3.2GB
    Database size After upgrade to vBulletin 5.1.6 are around 12G (excluding the old tables will be around 7G)
    upgrade to 5.1.7 still processing something related to visitor messages notifications (since long time, through cli). it seems it will never finish

    new closure table: ~ 21m records, 3.1GB
    new text table: ~ 3.7m records, 2.9GB
    new node table: ~ 3.8m records, 2.5GB

    The upgrade is done on test copy of the database.

    The problem, big database size will make it very difficult to backup, migrate, do everything regarding the database.

    I cannot live with this increase of size, which I think it is unnecessary. In fact, this will make me refrain from upgrading my real forum ( though I really loved vBulletin 5)

    otherwise. I need a solution (even if it required modification to vBulletin or whatever)

    keep in mind that I am very well in technical side. I can even manage to create my own "import script" if that can resolve the problem)

    thanks

    منتديات درر العراق
  • Trevor Hannant
    vBulletin Support
    • Aug 2002
    • 24325
    • 5.7.X

    #2
    Those tables are required by vBulletin 5 due to the new and changed structure from vBulletin 4. You need those, there's nothing other than deleting any 'tmp' tables and running optimze commands on your database you can do to reduce this.

    You are storing all attachments etc in the filesystem?
    Vote for:

    - Admin Settable Paid Subscription Reminder Timeframe (vB6)
    - Add Admin ability to auto-subscribe users to specific channel(s) (vB6)

    Comment

    • Oracle simsim
      Senior Member
      • Jul 2009
      • 117

      #3
      Originally posted by Trevor Hannant
      Those tables are required by vBulletin 5 due to the new and changed structure from vBulletin 4. You need those, there's nothing other than deleting any 'tmp' tables and running optimze commands on your database you can do to reduce this.

      You are storing all attachments etc in the filesystem?

      Thank you for quick answer. Yes I am using filesystem to store attachments, profile images, etc so the database one for textual information. the big size is mainly in those three tables. database overhead ~ 0 and no tmp tables (unless you consider the old post, thread tables are temp.

      so according to your experience in similar cases (i.e. large forum with vBulletin 5) what people do ?
      منتديات درر العراق

      Comment

      • tranceup
        Member
        • Jan 2005
        • 50
        • 5.0.X

        #4
        Originally posted by Oracle simsim


        Thank you for quick answer. Yes I am using filesystem to store attachments, profile images, etc so the database one for textual information. the big size is mainly in those three tables. database overhead ~ 0 and no tmp tables (unless you consider the old post, thread tables are temp.

        so according to your experience in similar cases (i.e. large forum with vBulletin 5) what people do ?
        we are in the same situation.
        Could you solve it?

        Comment

        • Wayne Luke
          vBulletin Technical Support Lead
          • Aug 2000
          • 73981

          #5
          Originally posted by tranceup
          Could you solve it?
          Due to the hierarchal structure of vBulletin 5's content, the database will be larger. The majority of unused tables should have been deleted during the upgrade process and thus freeing up some space. However, generally, the database will be larger. There is no workaround except to delete content.
          Translations provided by Google.

          Wayne Luke
          The Rabid Badger - a vBulletin Cloud demonstration site.
          vBulletin 5 API

          Comment

          • tranceup
            Member
            • Jan 2005
            • 50
            • 5.0.X

            #6
            Hi,

            On the DB, no, it is not normal but it makes it grow vBulletin. The last time I checked they confirmed that it was a Bug and that in a next update they would solve it.

            Before moving to vB5 it was at 6.5 GB, immediately after going to vB5 it was at 12 GB, and in less than 24 hours it ranged from 17 GB with peaks at 22 GB. The increase was spread over several tables, but the biggest one was always search log.

            Comment

            • tranceup
              Member
              • Jan 2005
              • 50
              • 5.0.X

              #7

              I pass in the form of a list the sizes of the largest tables

              search_log = 7.5GB
              node = 3.6GB
              text = 3.6GB
              closure = 2.5GB
              notification = 2.5GB

              The rest is below 1 GB

              Looking for previous tickets, I found the links I'll give them again



              Performance
              Updated the handling of the searchlog table in order to control its size. This will help prevent issues with building pages and creating new content.
              Added an external file that allows the running of Scheduled Tasks via the server's cron / scheduled task system. This will allow separation of these tasks from the web server.

              I mean it's a bug more than known and bleached by them

              Comment

              • Wayne Luke
                vBulletin Technical Support Lead
                • Aug 2000
                • 73981

                #8
                Is this issue causing performance problems on your site?

                If not, the actual size of the database tables isn't a big concern. However, you can truncate the searchlog table if it is a big concern of yours. You shouldn't do anything to the other tables you listed or you will destroy your content and it will not be retrievable.
                Translations provided by Google.

                Wayne Luke
                The Rabid Badger - a vBulletin Cloud demonstration site.
                vBulletin 5 API

                Comment

                • maxmobil
                  New Member
                  • Nov 2008
                  • 8
                  • 5.3.x

                  #9
                  I'm updateing a vB2 with a DB Size about 6GB without searchtables an it is increasing to 17GB to 20GB without searchtables. Is there any reason or explanation for this big (11-14GByte!!) overhead? We are talking about simple text in a DB, i dont think vB5 ist storing the content double, and even so i dont think the structure of vB5 is using so much content....?

                  Comment

                  • Wayne Luke
                    vBulletin Technical Support Lead
                    • Aug 2000
                    • 73981

                    #10
                    Have you completed the upgraded to 5.4.0? Earlier versions do double and triple store the content. We're not really concerned about database size as much as database speed.

                    I suspect that now that we have deprecated support for older versions of MySQL, we will start to take advantage of new tools that can reduce the database size over time.
                    Translations provided by Google.

                    Wayne Luke
                    The Rabid Badger - a vBulletin Cloud demonstration site.
                    vBulletin 5 API

                    Comment

                    • maxmobil
                      New Member
                      • Nov 2008
                      • 8
                      • 5.3.x

                      #11
                      Originally posted by Wayne Luke
                      Have you completed the upgraded to 5.4.0?
                      Yes, it is the 5.4.0.

                      Comment

                      • Wayne Luke
                        vBulletin Technical Support Lead
                        • Aug 2000
                        • 73981

                        #12
                        It is a complicated database. I wouldn't be able to tell you if the size is proper or not without looking at it. However, you cannot run vBulletin 5 without content in the search tables.
                        Translations provided by Google.

                        Wayne Luke
                        The Rabid Badger - a vBulletin Cloud demonstration site.
                        vBulletin 5 API

                        Comment

                        Related Topics

                        Collapse

                        Working...