[fixed] Update user post count problem

[David Bott]

Ver 2.2.2

Ok..here we go again... As mentioned in a past post when you go to mass purne out members it would only do like 50 out of thousands. They said..."this has been fixed in 2.2.2" so I waited to get 2.2.2 before trying again.

Well, I now have 2.2.2 and I am sorry to say it still does not work as it should.

When I went to prune out, I had 12,608 members on this one site I was working on. It found 1,031 members that met my search for the prune. (ANY, 365, 2001-11-01, 5) When it was all said and done with the prune, I eneded up with 12,237 members remaining. Thus only 371 out of 1,031 were removed.

A little while later I notice that this number had dropped to 11,837 making it 771 removed....This is very weird and was not a cache issue.

Now what makes this even worse is...If I re-run the same search to remove members...I end up with it only finding 101 members. Where did the other 159 go for the numbers really, really do not add up at all. (ok you do all the math...all in all it is messed up.)

Can you follow all that? Hope so...for this is now a major issue for I am not sure what is happening to this data when you are doing this mass purne of members for the numbes do not compute.

[David Bott]

Ok...Lets follow this up now. I went to try the so called remaind 101 members from the search.

11,838 Starting total member count
101 found from the mass prune search (ANY, 365, 2001-11-01, 5)
Told it to DELETE all found.
11,787 Total Members after prune.
Seems Only 51 were removed leaving 50.
Rerun the serach...50 found!
Tell it to delete the found 50.
11,737 members now remaining. (I still have a lot more to go.)

Note...the numbers are still off from the origional 1,031 found members in my origional serach...this still no clue why the counters are so messed up.

[mvigod]

Just tried to update user post counts and noticed a user who had complained his count was wrong wasn't indeed counting...upon inspection one of the forums he posted to was not set to include post counts so I changed it and reran the update post counter.

After running it the post count still didn't change so I changed manually to a bogus number and then reran the update counters but still no change...

More troubleshooting and some manual SQL queries showed that he indeed had 19 posts but was only still showing the 15 bogus count after running update counters.

I then looked at the update counter code and decided instead of running 1000 users at a time (he was userid 2801) to run 5000 at a time to cover all 3500 users in the first pass of the update instead of the multipage redirect when using 1000 at a time. Well...that did the trick...I think only the first pass is updating user posts counts and each redirect pass does nothing to anybody's post counts.

Is this a bug? Could somebody see if they can replicate this? Change a users post counts to a bogus number in the user table and then run the update counters to make sure it updates them. Set your number of users to a lower number than the userid of the user you just changed so the script trys to change it on a redirect instead of on the first pass.

Marc

[mvigod]

Looks like a bug.

The action on the redirect is incorrect in 2.2.1.

It should be action=updateposts and not action=updateuser under the updateposts section of misc.php


if ($checkmore = $DB_site->query_first("SELECT userid
FROM user
WHERE userid>=$finishat")) {
cpredirect("misc.php?s=$session[sessionhash]&action=updateuser&startat=$finishat&perpage=$perpage");
echo "<p><a href=\"misc.php?s=$session[sessionhash]&action=updateuser&startat=$finishat&perpage=$perpage\"
>Click here to continue updating users</a></p>";


marc

[Freddie Bingham]

I'll look at this in a bit and see what is really going on there. If I recall I believe 50 users are deleted than the page redirects and deletes another 50 and so on until it is finished. At least that is how it is supposed to work.

[David Bott]

Hi...

Yes, suppose to. So you know...it does go through the entire list...page by page. It just does not delete the ones after the first page it seems. But the numbers do not jive. Remember, I say it did seems to delete 371 members the first time.

It is all very strange. Thank you for taking the time to look. FYI..I have a lot more to remove, so we do have a test bed as long as it will not destroy my member base.

[Freddie Bingham]

Replace your admin/user.php with this one and let me know the outcome.

[David Bott]

Hi...

OK, so far looking good.

I did the serach for 2001-11-01 and less than 6 pots.

Started with 11,795
Found 264
After Delete - 11,531 (Which is correct!!!)

Did it again...

I did the serach for 2001-11-01 and less than 8 pots.

Started with 11,531
Found 369
After Delete - 11,162 (Which is correct!!!)


So, this is looking to be very good. I will give it a larger delete test later on for I need to get to my other office right now. (Boy, can't wait to see if you can fix the other issues I have which I will report alter to you. :))

[David Bott]

Ok...Not sure...But I wanted to make mention of something so you can look into it and advise...

After doing the above deletions the WEB SERVER side of my setup started running REALLY high.

I had made the deletions for the TiVo forum...So, I closed that forum down. I left the large AVS Forum open with all 800+ users on it. The server load droped down to where it should be. (with TiVo open...25+ all day...with it closed...2.5)

I reopened TiVo and the load started to climb again. I then closed it and it fell. Open and it climbed.

So...I then shut down MySQL and ran a complete repair on the TiVo database. Reopened both sites and my load it again where it should be.


So...My question is...can you please look to see if you find anything in that deletion sequence that might have caused a DB issue.

I have tried any more deletions as of yet.

Thank you.

[Freddie Bingham]

Deleting users is a mysql intensive process as for each user you delete, a query had to be ran on the thread table and a query has to be ran on the post table. This version is more query firendly than the previous version though by a bit. I can't give you any reasons why a server would continue to experience high load after the process is completed though.

[Floris]

Originally posted by freddie
Replace your admin/user.php with this one and let me know the outcome.

I don't have a big forum, or prune a lot .. but I do think it is wise, to stay up to date with the latest 'working' version.

So it is wise, for me too, to download that file and replace it?

[Omero]

Originally posted by mvigod
Just tried to update user post counts and noticed a user who had complained his count was wrong wasn't indeed counting...upon inspection one of the forums he posted to was not set to include post counts so I changed it and reran the update post counter.

After running it the post count still didn't change so I changed manually to a bogus number and then reran the update counters but still no change...

More troubleshooting and some manual SQL queries showed that he indeed had 19 posts but was only still showing the 15 bogus count after running update counters.

I then looked at the update counter code and decided instead of running 1000 users at a time (he was userid 2801) to run 5000 at a time to cover all 3500 users in the first pass of the update instead of the multipage redirect when using 1000 at a time. Well...that did the trick...I think only the first pass is updating user posts counts and each redirect pass does nothing to anybody's post counts.

Is this a bug? Could somebody see if they can replicate this? Change a users post counts to a bogus number in the user table and then run the update counters to make sure it updates them. Set your number of users to a lower number than the userid of the user you just changed so the script trys to change it on a redirect instead of on the first pass.

Marc

I was about to post a bug report on this! It's exactly the same that i found: USER POST COUNT DONT GET UPDATED AT ALL!

I will try your "all in a pass trick" and see if it's there the problem, but one thing is for sure: the update postcount doesn't work as it should.

[Omero]

I checked it and mvigod was indeed right! The redirection is wrong, changing it from action=action=updateuser to action=updateposts fixed it!

I tried putting a bogus number as a "high ID" member number of posts (so it doesn't get into the first run of the script), and run the script with 1000 each run.

It did got updates, as opposed to remain the same before this little fix ;)

Wow you did it :D

[Freddie Bingham]

It still won't always work right unless you use the updated file I provided above.

[Omero]

Originally posted by freddie
It still won't always work right unless you use the updated file I provided above.

I did both ;) Using new user.php and updated misc.php myself ;)

[ubbuser]

Hmm, so misc.php requires an update also?? :confused:

[Floris]

Originally posted by ubbuser
Hmm, so misc.php requires an update also?? :confused:

I feel like calling my vB 2.2.2,1 now :)

[David Bott]

I will make mention again that the load was high WELL AFTER the deleting was done. All day infact until I ran a fix. It was not a high load only during the process.

I will do some more user deletes....if the load is again messed up, something is wrong.

[ubbuser]

I downloaded the new user.php and did a mass prune for users, and now this post count for existing users is totally hosed... :confused:

[David Bott]

okay...did another prune add all seems fine. So, I will stay it must have been a fluke.


Not sure about your hosed user post counts...I had no such issues. Did you manybe run a reset (recount) on them? I ask this because it does not do a recount during that process that I am aware of.

[Jet]

Originally posted by xiphoid


I feel like calling my vB 2.2.2,1 now :) Sorry but didn't understand which are the minor mods to make the minor version upgrade (0 to 1) ... :D
Could you post them here or even a checklist ?
I have a similar problem which seems not having had solution for weeks, here (http://www.vbulletin.com/forum/showthread.php?s=&threadid=38372) ...
Thanks a lot.
Bye