Suggestion: Forum for Large/Multi-Server VB implementations

[bigsoccer tech]

Pretty please...

[Brains]

I'm in... Not to be an "elitest" or anything, but I'd also like to see an appropriate forum description added so people with 50,000 posts don't think their forum is "large" and lower the s/n ratio. Maybe 2 million+ posts, and 500+ concurrent users? Thinking back, that's when we had to start thinking in new ways about resource utilization.

Maybe even be a posting-restricted forum? VB staffers and large forum owners/admins have posting permission, but still publically viewable.

[David Bott]

AVS Forum and TiVo Community Forum would like to concure. We large Vb users really do need a place to chat on topics that are more of an issue to large operations of the Vb platform.

Thank you kindly for the consideration. (Would be nice to get out of it all being in mostly one thread. :))

[The Prohacker]

I agree completely that there needs to be a forum dedicated to larger forums and their issues.

[Karri]

I would like to see this as well. Thanks.

[Colin F]

I'll bring this up for discussion among staff...

[RedWingFan]

Add my vote as well. I'd regularly browse such a forum area to see if there are any tips I could pick up. Much easier than wading through a lot of issues that come up on smaller forums and not knowing if they apply.

[ManagerJosh]

Or maybe even some codehacks that can help achieve better multi-server implementations.

[RedWingFan]

Or maybe even some codehacks that can help achieve better multi-server implementations.
A companion "Large Forums" section over at vb.org would be ideal. I know how the vB staff feels about modifications to the code--it would be their call on what they would/would not allow in terms of modified code in this section. Good thing is, if posted here, they might see one of our hacks and implement it in the final version.

I'm in... Not to be an "elitest" or anything, but I'd also like to see an appropriate forum description added so people with 50,000 posts don't think their forum is "large" and lower the s/n ratio. Maybe 2 million+ posts, and 500+ concurrent users? Thinking back, that's when we had to start thinking in new ways about resource utilization.
We hit our brick wall on our old dedicated server when we were around 1 million posts, but only had maybe 350-400 users online. We're on the new server now, but hitting the wall with 1.5 million posts and 500-600 users online. We're tweaking things and considering archiving old posts, which will help, but I feel that once you're on a dedicated server, you're already "big board" material, in that you're using too many resources to be in a shared hosting environment, and you forsee growth potential. With dedicated hosting, too, we have more options available to configure our own settings (like my.cnf), where under a shared hosting arrangement, we are pretty much locked into standard settings our host offers us.

IOW, we may not want hard limits to determine what constitutes a larger forum, but you're right--we also don't want someone with a small forum on a shared server posting here, even if the reason is that there is not much we can offer in the way of advice!

[kerplunk]

I would love to see this forum added also.

[alexi]

I think this is a great idea

[Creepshow]

Good idea. Count my vote.

Although I'm not holding my breath. ;)

[bmanz]

great idear.
and i vote yes

[KrON]

#4 chimes in with a pretty please! :)

[Scott MacVicar]

John has brought the topic up and we're still trying to decide what to do, should we provide a private forum for larger boards and post code hacks / modifications to let them try performance things out or a public forum.

The problem with the public forum is that posting changes would violate our own posting rules of no code hacks :)

Maybe a public forum and then a sub forum for modifications.

[RedWingFan]

John has brought the topic up and we're still trying to decide what to do, should we provide a private forum for larger boards and post code hacks / modifications to let them try performance things out or a public forum.

The problem with the public forum is that posting changes would violate our own posting rules of no code hacks :)

Maybe a public forum and then a sub forum for modifications.
Advantage of a public forum: someone who's on the verge of having a large forum could browse posts to see how others are dealing with growth issues. IOW, someone could be on a shared hosting account, using too many resources, and is curious about how the admins of larger forums have their forums configured. That could be helpful when trying to move up to a dedicated server, for instance.

Disadvantage: the posting of code hacks which goes against your forum rules.

One idea might be to have a Large Forums usergroup: you could give read-only access to everyone, but give posting and attachment-viewing privileges only to members of the group. Or to expand on your idea: the sub-forum with the actual hacks could be viewable only by members in this group, where discussions in the parent group could still be read-only by all members.

I don't know what criteria I'd use, but I would think that anyone who's on a dedicated server should have access. I wouldn't want to stick to statistics only: a forum could have only 500,000 posts, but may be extremely popular early on with 700-800 members online at any one time. I don't know if anyone on the vB staff would have time to check out new memberships in the usergroup, or just do it on the honor system.

[Slapyo]

While I don't qualify for having a large forum, I do find reading the large boards thread very interesting. I for one would definitely like to see it a public forum.

What about a large board forum on vB.org where code hacks could be posted?

[daemon]

While I don't qualify for having a large forum, I do find reading the large boards thread very interesting. I for one would definitely like to see it a public forum.
I agree with this sentiment. Not only as a forum owner who's curious about larger operations, but as a software developer that wants to see what can be done to make web applications more scalable.

[encryption]

I personally would like to see a forum for the larger boards too, not because mine is anywhere near as large, but from a webhosts perspective, it would help to learn about the "issues" that large boards can / do face and the various "fixes" out there being implemented.

You could probably setup a disclaimer saying that any code modifications obtained from this forum / subforum are at your own risk and vBulletin will not provide any support or guarantee to the effectiveness or peformance of the code tweaks. This way you guys can stay in the clear and probably even get some useful info from the gurus of the larger boards.

[Bendo]

John has brought the topic up and we're still trying to decide what to do, should we provide a private forum for larger boards and post code hacks / modifications to let them try performance things out or a public forum.

The problem with the public forum is that posting changes would violate our own posting rules of no code hacks :)

Maybe a public forum and then a sub forum for modifications.

A public forum would be a good thing to have because none vB users could at least see what problems current users have and determine if vB would be suitable for their needs. vB already leads the way with this over IPB, IMO since IPB hide absolutly everything behind closed doors meaning one cannot even get a sence of it's suitability. Public access would at least help people considering converting from other software assess whether moving to vB would be suitable for their needs.

Thats my view as a none vB user, feel free to ignore it or whatever :).

[Nombie Wan]

Count me in. I think this would benefit Jellsoft and the community of large boards.

[john1744]

What are some general ideas for improving server performance on a forum with 1.25 million posts and about 150 users online in the afternoon.

We have a single dedicated server with a P4. Our site begins to choke when about 100 users are online.

Is it possible to offload the mySQL to another server and keep the forums on another?

Or are there features I should turn off to improve performance?

[ManagerJosh]

What are some general ideas for improving server performance on a forum with 1.25 million posts and about 150 users online in the afternoon.

We have a single dedicated server with a P4. Our site begins to choke when about 100 users are online.

Is it possible to offload the mySQL to another server and keep the forums on another?

Or are there features I should turn off to improve performance?
If you look in config.php, you could change the server from localhost to another server, provided that you do have another server dedicated to handling MySQL.

[bigsoccer tech]

So any verdict? the 1mm vb thread is so hard to sort thru...

[Steve Machol]

If you have a problem, start your own thread in the server configuration forum. :)

[iardon]

I also don't have a large forum but would be interested in at least reading the posts. So please make it public if this happens.

[TheMusicMan]

Not a bad idea actually.

I am not at that stage just yet, but it would be nice to be able to read up a little about the more advanced aspects of multi-server management, and the technical aspects of running larger vBulletin forums. I am aware though for those needing information now, that there are already such places where these type of dicsussions take place. I am sure theAdminZone has a forum category for admins of Big-Boards - perhaps take a look there and see if this meets your needs for discussion?

Just 140,000 posts (and counting) to go for me to be able to start posting there...:):)

[Shining Arcanine]

A forum for this would be an interesting read. Please consider having one.

[Jpax]

My board is about to hit the 1 million mark, with 200~400 active members on between 10am and 11pm ,I would like to have a place to talk about how to make my fourm run better.

[Phrost]

Pretty please...

The Admin Zone (not my site) has an area exclusive for "big board" operators as judged by big-boards.com. I think there are some Vbulletin staff over there too.

Anyway, it's been a good resource for me as I run a big board.

[thegunowner149]

The Admin Zone (not my site) has an area exclusive for "big board" operators as judged by big-boards.com. I think there are some Vbulletin staff over there too.

Anyway, it's been a good resource for me as I run a big board.
I concur. I own a big-board too and have found the admin zone (www.theadminzone.com/forums) to be the best source of vBulletin related discourse around.

The big-board forum is screened and accessible via usergroup permission only which is nice.

[Reeve of Shinra]

www.theadminzone.com & www.best-boards.com are both good resources for forum admins but I would still like to see an official vb forum.

[rex_b]

www.theadminzone.com (http://www.theadminzone.com) & www.best-boards.com (http://www.best-boards.com) are both good resources for forum admins but I would still like to see an official vb forum.

I concur, and www.theforumfix.com (http://www.theforumfix.com) is great resource as well.

[MGM]

I would also like to suggest theadminzone.com ... I am a regular member there and have found it to be a great resource...

www.theadminzone.com (http://www.theadminzone.com/) & www.best-boards.com (http://www.best-boards.com/) are both good resources for forum admins but I would still like to see an official vb forum.

What do you mean? I thought we were on the official forum right now :p

MGM out

[Reeve of Shinra]

The forum fix is good, I like the articles about making money LOL


What do you mean? I thought we were on the official forum right now


LOL I meant a forum on vb.com for owners of larger boards. I would love to try out tweaks that came directly from the dev team!

[woodysfj40]

just requested a username for theadminzone...looks like a great resource...

500ish online users, 2500+ posts a day, already on dual servers...mine'll break 1m posts in a few weeks....

[IDN]

I don't have a 1mil+ forum, but I vote a forum for large boards and hacks over at vB.org if they can't be put here.

[Scribbller]

I would also recomend the Big Boards forum on www.theadminzone.com the best admin resource hands down, there are other smaller ones but TAZ is the fastest I say.

[Colin F]

I would love to try out tweaks that came directly from the dev team!

If such a forum were set up, it would be primarily for interaction between customers with big boards. As some of the adjustments being used now and probably in the future require the code to be modified, we can't support them.

[eva2000]

skipping over where to host this forum on vBulletin.com or .org or elsewhere, the idea itself is excellent

who's to say whether through discussions between large forum operators we arrive at fixes, and solutions to troublesome issues which can be implemented in the next version of vBulletin or come across hardware compatibility issues and the like we can be aware of i.e. centOS + 2.6.9-22 kernel bug http://www.vbulletin.com/forum/showthread.php?t=171071 :)

[Slapyo]

The Admin Zone (not my site) has an area exclusive for "big board" operators as judged by big-boards.com. I think there are some Vbulletin staff over there too.

Anyway, it's been a good resource for me as I run a big board.
So that forum is available only to people who are listed on big-boards.com? I'd like to be able to read that forum, but don't have a big board. :/

[iardon]

I agree. I don't understand why you have to have a "big board" in order to gain knowledge about how to run one in the future? :confused:

So that forum is available only to people who are listed on big-boards.com? I'd like to be able to read that forum, but don't have a big board. :/

[the Sandman]

LOL - We had this same conversation when we opened the Big-Boards Forum. For the most part, topics discussed it that forum only apply to big boards. And, like here, the administrators of the big boards post all over the place, not just within that forum. But they do like to have a private place to discuss topics of interest to Big Board admins. Premium members of the Admin Zone who do not run big boards are able to see the discussions but cannot post in them. Among other things, posting an article will get you into the Premium group.

[Reeve of Shinra]

If such a forum were set up, it would be primarily for interaction between customers with big boards. As some of the adjustments being used now and probably in the future require the code to be modified, we can't support them.

Naturally ;)

[Carnage-]

I think it could be a good read. I think our forum counts as big 200-600 online can get as meny as 1500 at times, only at 650k posts at the moment however we do have nearly 20000 registered users. Its running on a mostly dedicated server; thers a couple of other sites on their and we run irc and cs servers it does get very slow at times for no apparent reason.

This forum may help me tweak it to be a little faster. Then again i'm sure the upgrade to vb3.5 will help a bit.

[Razasharp]

Theres such a section at TAZ which is a great 'unoffical' place to go :)

However I think such a section here would be very beneficial, to help the vB team better understand big board admins needs.

[Erwin]

I am happy to open such a forum at vBulletin.org (where I am admin) pending approval from Jelsoft of course.

[Razasharp]

No offence Erwin, It's kind of you to offer, but I think it would be nice to have an outlet where prominent vB users can talk directly to the team - if it were on vB.org I don't think things raised would get across to the team as much if say, if it were here.

[Nombie Wan]

The info a forum like this would provide for Jellsoft is reason enough in my mind.

I really don't understand why they don't want to open one. There are a LOT of vB's with > 5 million posts these days, and it's really a whole different app at that level.

[Swamper]

I agree that this is long overdue!

I'm in the process of moving from vb2.3 to 3.5.4 and re-doing all my hacks... the biggest headache will be to get search as good as I've customized it on vB2 (5.5+ Million posts)

[Rifleman000]

Agreed, a public accessible, but not writeable forum here would be a great addition for boards already breaking at the seems of current technology to develop further.

A mark would have to be established for entry though, such as 1 million posts, or by posts per week, or some other concrete number for admins to gain access.

[LeeD]

Agreed, a public accessible, but not writeable forum here would be a great addition for boards already breaking at the seems of current technology to develop further.

A mark would have to be established for entry though, such as 1 million posts, or by posts per week, or some other concrete number for admins to gain access.
Agreed. I would think it could be available to boards with more than 500,000 posts maybe. I would definitely want to be involved as I have three forums of that size wih a fourth on the way, and I am continually working on speed and trying to prepare for the continued growth.

[eva2000]

well my definition of large http://www.vbulletin.org/forum/showthread.php?p=921784#post921784

It's not just size in terms of number of posts, but activity level as well particularly the concurrency level of the forums. i.e. 10,000,000 post forum with only 5 members activity using the forums and searching would have no issues compared to 1,000,000 post forum with 2,500 users trying to view and reply to the same thread at the very same time or 2,500 users trying to search the forums at the very same time - it's all about concurrency for me :)

vB users online is hard to guage due to cookie timeout out and can only be used as guide in combination with web stats of unique visitor counts as well as the number of mysql concurrent connections to the mysql server.

Personally i'd consider

- a medium large forum as any forum which has 500+ vB users online over default cookie timeout and at least 300-400 mysql concurrent connections (this is usually the threshold where most vB forums are forced to split to more than one dedicated server)
- a large forum as 800+ vB users online over default cookie timeout and at least 400-600 mysql concurrent connections
- a very large forum as 1,000+ vB users online over default cookie timeout and greater than 600+ mysql concurrent connections

[LeeD]

Very interesting insight George. :)

[telc]

I am interested in this also.