vB-5.3.4 PL1: email notification jobs running sometimes only

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • m97forum
    Member
    • Jun 2017
    • 54
    • 5.3.x

    [Bug / Issue] vB-5.3.4 PL1: email notification jobs running sometimes only

    Hello,

    our vBulletin installation (version 5.3.4 Patch Level 1) has the following email notification jobs being set up:
    1. daily email notification (should run daily at 15:02 CET and write a log entry)
    1. weekly email notification (should run every Monday at 0:30 CET and write a log entry)

    Both jobs work once in a while and their emails are received by the users.

    BUT: the job just tends not to run. Here's the log, when the
    A. daily job ran over the past couple weeks (copied from the task's log):
    07.02.2018 17:17
    07.02.2018 01:50
    05.02.2018 18:02
    04.02.2018 20:32
    04.02.2018 01:04
    30.01.2018 18:20
    27.01.2018 17:10
    26.01.2018 16:35
    23.01.2018 17:45
    18.01.2018 16:26
    16.01.2018 18:15
    15.01.2018 16:36
    14.01.2018 21:47
    14.01.2018 01:31
    10.01.2018 17:58

    ... as you can see, sometimes the job runs several times a day, but never at the scheduled time. Sometimes, it does not run at all. Looking at the new Thread/Posts statistics, we've had new posts every day of the list above.


    B. weekly job ran over the past couple weeks (copied from the task's log):
    01.01.2018 11:57
    09.10.2017 07:08
    02.10.2017 10:22
    25.09.2017 09:41
    18.09.2017 10:48

    ... as you can see, the job ran just once(!) in 2018 and before that in October 2017 for the last time (we switched from vB-3.8.7 to vB-5.3.3 in mid-October, before that everything seems to have been fine). And it has never run at the time it was scheduled to do so.

    Any ideas, what may be the cause for this? The hosting provider has not yet informed me from their side, that they would shut the system down just in time for the scheduled job...

    Thanks in advance.

    Stefan
  • Wayne Luke
    vBulletin Technical Support Lead
    • Aug 2000
    • 73976

    #2
    Scheduled Tasks require traffic to the site. Only one task is run every minute and only one per end user. If there isn't constant traffic then they can fall behind. There is a file in the do_not_upload folder where you can set up the scheduled tasks to be called from your server's cron/task manager for more reliable usage. It is called vbcron.php. The instructions are in the file. Look at it in a text editor.
    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud demonstration site.
    vBulletin 5 API

    Comment

    • m97forum
      Member
      • Jun 2017
      • 54
      • 5.3.x

      #3
      Hello Wayne,

      thank you for giving me this hint.
      I just set the cron job up.
      Now I'm receiving the following info:

      X-Powered-By: PHP/5.6.30 Content-type: text/html; charset=UTF-8 cannot use vB CLI scripts from the web
      Is that a valid answer or is there something missing?

      Thanks.
      Stefan

      Comment

      • Wayne Luke
        vBulletin Technical Support Lead
        • Aug 2000
        • 73976

        #4
        You shouldn't be calling that script from the web. It should be outside your public_html directory and should be called from the command line: php /path/to/file/vbcron.php (note the space between php and the first /)
        Translations provided by Google.

        Wayne Luke
        The Rabid Badger - a vBulletin Cloud demonstration site.
        vBulletin 5 API

        Comment

        • m97forum
          Member
          • Jun 2017
          • 54
          • 5.3.x

          #5
          Alright, I've set the Cronjob to run every 5th minute, which is the minimum allowed by my hosting provider.
          Yesterday, on the Feb. 14th, a total of 3 new posts had been created by 15:02 CET (when the dialy digest is launched).
          Still, the job did not run (I think, it should have run anyway, maybe finding no post to publish).

          The hosting provider's cronjob gets the following result:

          Zuletzt ausgeführt: 15.02.2018 um 13:50
          X-Powered-By: PHP/5.6.30 Content-type: text/html; charset=UTF-8

          ... which looks good to me.
          It seems, that I do not hit the exact time, when the digest expects/needs it.

          1. how exact does the hosting provider's Cronjob (which runs at 15:00 and 15:05) need to hit the time set for the digest (which is scheduled for 15:02)?
          2. is there any other way to extend the time frame during which the hosting provider's Cronjob may hit the digest?
          3. as long as we used our previous version vB-3.8.1 (not vB-3.8.7 as written above), the digest ran each day without any troubles. What has changed here giving us that trouble now?
          4. I have a private Owncloud box running at home. Although I dont like to mix things up, I can setup a Cronjob there to connect to the vB in order to provide some traffic. What would I need to do for that to happen? It is an external box, so is it enough to just run a curl on the vB homepage?

          Thanks a lot.

          Stefan

          Comment

          • Wayne Luke
            vBulletin Technical Support Lead
            • Aug 2000
            • 73976

            #6
            You could be affected by this bug: http://tracker.vbulletin.com/browse/VBV-13074

            Instant Notification should be the preferred method.

            To answer your questions:
            1) it doesn't need to be on the time at all. The system works by queuing tasks and one task will be run every five minutes. The times in the database are when the task is added to the queue.
            2) We recommend running the vbcron.php file every minute. Since your provider doesn't allow this, you are stuck with the time frame already configured.
            3) vBulletin 3.X is different software with different processes. Not only did vBulletin 4 rewrite a significant portion of vBulletin 3 processes over time, vBulletin 5 has replaced more than 80% of the vBulletin 4 code with new and modern code. We really can't compare the different versions.
            4) The vbcron.php is designed to be run on the same server and uses a method that bypasses remote authentication. This would not be secure running it from another box. Using cURL does not trigger scheduled tasks. That is why we had to add the file.
            Translations provided by Google.

            Wayne Luke
            The Rabid Badger - a vBulletin Cloud demonstration site.
            vBulletin 5 API

            Comment

            • m97forum
              Member
              • Jun 2017
              • 54
              • 5.3.x

              #7
              Hello Wayne,

              I'm still stuck on this. The notification message are send, but quite rarely.
              Please explain to me the following:

              "1) it doesn't need to be on the time at all. The system works by queuing tasks and one task will be run every five minutes. The times in the database are when the task is added to the queue."

              ... what happens, if I set up the notification job to run at say 15:02, but my vbcron.php is run at 15:00 and 15:05? Is the job executed then anyway? Or does the timing of the vbcron.php need to fit the time set for the notification job in vB?

              And do I get you right, that I cannot provide some traffic to our vB installation using an external call (that's what I meant using a curl-call from my private remote machine)? It provide a call to vB every minute. But I don't know, if the job needs an authenticated user, rather than a guest user.

              Thanks.
              Stefan

              Comment

              • Wayne Luke
                vBulletin Technical Support Lead
                • Aug 2000
                • 73976

                #8
                Originally posted by m97forum
                Hello Wayne,

                I'm still stuck on this. The notification message are send, but quite rarely.
                Please explain to me the following:

                "1) it doesn't need to be on the time at all. The system works by queuing tasks and one task will be run every five minutes. The times in the database are when the task is added to the queue."

                ... what happens, if I set up the notification job to run at say 15:02, but my vbcron.php is run at 15:00 and 15:05? Is the job executed then anyway? Or does the timing of the vbcron.php need to fit the time set for the notification job in vB?

                And do I get you right, that I cannot provide some traffic to our vB installation using an external call (that's what I meant using a curl-call from my private remote machine)? It provide a call to vB every minute. But I don't know, if the job needs an authenticated user, rather than a guest user.

                Thanks.
                Stefan

                At 15:00, vBulletin will look at the Scheduled Task Queue and run the next one in line. At 15:05, it will look at the scheduled task queue and run the next one in line. At 15:10, it will look at the scheduled task queue and run the next one in line.

                At 15:02, your email notification scheduled task will be added to the end of the queue. If it is the next one in line, it will be run at 15:05. If there are 100 scheduled tasks in the queue already, it becomes number 101. It doesn't get to jump the line.

                Which task this is will vary on every single vBulletin installation and is determined how many scheduleded tasks are in the queue. You don't need to simulate traffic. You're using the vbcron.php which triggers a scheduled task every time it is run. We recommend running it every minute.
                Translations provided by Google.

                Wayne Luke
                The Rabid Badger - a vBulletin Cloud demonstration site.
                vBulletin 5 API

                Comment

                • m97forum
                  Member
                  • Jun 2017
                  • 54
                  • 5.3.x

                  #9
                  Hey again,

                  I just updated to vB-5.4.0. I kept the vbcron.php of 5.3.4-PL1 (I tried vbcron.php of vB-5.4.0 as well).

                  But now I get the following result:

                  Die Ausgabe ist auf 1024 Zeichen begrenzt und wird nach 80 Zeichen automatisch umgebrochen, um die Lesbarkeit zu erhöhen.

                  Zuletzt ausgeführt: 24.03.2018 um 20:58
                  X-Powered-By: PHP/5.6.31 Content-type: text/html; charset=UTF-8 cannot use vB CLI scripts from the web What's wrong here? Except for the vB-5.4.0 update, no change has occurred.

                  Thanks.
                  Stefan

                  PS: I don't know, if this forum here sends out notification emails at all - but I've never received anything, although I've subscribed to this thread here.

                  Comment

                  • delicjous
                    Senior Member
                    • Mar 2014
                    • 499
                    • 6.X

                    #10
                    Hello m97forum,

                    I have a how to use the cron.php in german here: vbulletin-forum.de

                    Comment

                    • m97forum
                      Member
                      • Jun 2017
                      • 54
                      • 5.3.x

                      #11
                      Hello again,

                      alright - I did some debugging today (basically I added lots of cronlog - inserts, since I do not get the Exception thrown printed anywhere).
                      Here are my finding:
                      1. the dailydigest is run by cron, not by an admin using the UI (keep that in mind for a minute)
                      2. in file core/includes/functions_digest.php:270 the channel permissions are requested, using the following call (which is part of a negated if-clause):
                      $usercontext->getChannelPermission('forumpermissions', 'canview', $thread['nodeid'])
                      here, $thread['nodeid'] evaluates to null or empty. Since there is no check for this value, it is given without any trouble to function getChannelPermission($permissiongroup, $permission, $nodeid, $parents, $parentid) of file core/vb/usercontext.php:759 ff.
                      3. since the caller is a superadmin, function getChannelPermission(...) hands this parameter on to function getChannelLimitPermission() of file core/vb/usercontext.php:821.
                      4. function getChannelLimitPermission() finally tries to load the actual node using the folllowing call on line core/vb/usercontext.php:533:
                      $node = vB_Library::instance('node')->getNode($nodeid);
                      5. function getNode($nodeid) in file core/vb/library/node.php:944 calls getNodeBare($nodeid) (while $nodeid is still null/empty)
                      6. function getNodeBare($nodeid) in file core/vb/library/node.php:867 finally throws a vB_Exception_Api('invalid_node_id'), since $nodeid is empty

                      In short: the question now is: why does $thread['nodeid'] in core/includes/functions_digest.php:270 evaluate to empty? Which data do you need to check?

                      Thanks a lot in advance.

                      Stefan
                      Last edited by m97forum; Fri 30 Mar '18, 1:50pm.

                      Comment

                      • m97forum
                        Member
                        • Jun 2017
                        • 54
                        • 5.3.x

                        #12
                        Hey there,

                        I just continued my search and I found something.
                        As far as I get it, file core/includes/functions_digest.php:255 loads the threads that fit into the given lasttime-setting. The query used is as follows (taken from core/vb/db/mysql/querydefs.php:2427):

                        Code:
                        [COLOR=#008000][B]'fetchForumThreads' [/B][/COLOR]=> [COLOR=#000080][B]array[/B][/COLOR](    vB_dB_Query::[COLOR=#660e7a][B][I]QUERYTYPE_KEY [/I][/B][/COLOR]=> vB_dB_Query::[COLOR=#660e7a][B][I]QUERY_SELECT[/I][/B][/COLOR],    [COLOR=#008000][B]'query_string' [/B][/COLOR]=> [COLOR=#008000][B]" [/B][/COLOR][COLOR=#008000][B]      SELECT forum.htmltitle AS forumhtmltitle, thread.nodeid AS threadid, thread.routeid, thread.htmltitle, [/B][/COLOR][COLOR=#008000][B]         thread.publishdate, thread.parentid AS forumid, thread.lastcontent AS lastpost, thread.open, thread.textcount AS replycount, [/B][/COLOR][COLOR=#008000][B]         thread.authorname AS postusername, thread.userid AS postuserid, thread.lastcontentauthor AS lastposter, thread.publishdate AS dateline [/B][/COLOR][COLOR=#008000][B]      FROM {TABLE_PREFIX}node AS thread [/B][/COLOR][COLOR=#008000][B]      INNER JOIN {TABLE_PREFIX}closure AS closure ON ( closure.child = thread.nodeid ) [/B][/COLOR][COLOR=#008000][B]      INNER JOIN {TABLE_PREFIX}node AS forum ON ( forum.nodeid = thread.parentid) [/B][/COLOR][COLOR=#008000][B]      WHERE closure.parent = {forumid} AND closure.depth >= 1 AND [/B][/COLOR][COLOR=#008000][B]         thread.nodeid = thread.starter AND [/B][/COLOR][COLOR=#008000][B]         thread.lastcontent > {lastdate} AND [/B][/COLOR][COLOR=#008000][B]         thread.showpublished = 1 [/B][/COLOR][COLOR=#008000][B]      " [/B][/COLOR]),
                        In the following, fields defined by the query may later be accessed using the "column" names of the result set, e.g. "lastpost". Now, core/includes/functions_digest.php:270 tries to load the nodeid, using $thread['nodeid'] - but "nodeid" is not defined as a result set column while "threadid" is.

                        I just extended the query to return thread.nodeid (which is column node.nodeid) by the name "nodeid". Suddenly, the cron-run dailydigest job gets executed (disclaimer: email sending is disabled during this debugging phase and I had to adjust the lastcontent-timestamp to include the last two day's posts).

                        If my conclusion is correct (which I now ask you to verify), I cannot understand how this entire dialydigest.php may be working at all. It just misses a column, leading to a crash in the following. So I'm not really convinced of my findings.

                        Any help is appreciated.

                        Stefan

                        Comment

                        • Wayne Luke
                          vBulletin Technical Support Lead
                          • Aug 2000
                          • 73976

                          #13
                          This is a known issue that is on our todo list - http://tracker.vbulletin.com/browse/VBV-13074
                          Translations provided by Google.

                          Wayne Luke
                          The Rabid Badger - a vBulletin Cloud demonstration site.
                          vBulletin 5 API

                          Comment

                          • m97forum
                            Member
                            • Jun 2017
                            • 54
                            • 5.3.x

                            #14
                            Ok, thanks for telling me ... is it going to be resolved some time? It was opened in 2014...

                            also: is it correct to use the thread's node.nodeid for this purpose here? It was just the first thing I could come up with. But I cannot tell, which field is supposed to be used here ... ?

                            Thanks.
                            Stefan

                            PS: why is the tracker not https.//-encrypted?

                            Comment

                            • m97forum
                              Member
                              • Jun 2017
                              • 54
                              • 5.3.x

                              #15
                              Hi there,

                              since it is still not working (I seem to have another set of data...), I was debugging again.

                              This time, it tries to load the postings content in line dailydigest.php:189. There it crashes w/ "no_permission" exception again.

                              I was able to trace it until famous usercontext.php.getChannelLimitPermission(...) (lines 504ff.). I cannot tell, which permission it exactly needs. But I have found out using $usercontext->fetchUserId(), that it runs using userId = 0. There is no user of that id in my user table.
                              Is that a special user? Which user is supposed to be running the daily-/weeklydigest?

                              Please note: I tried loading some page using another user and I that user's id - which is fine, because I therefore did not use some illegal statement here.

                              Thanks a lot.

                              Stefan

                              Comment

                              Related Topics

                              Collapse

                              Working...