New way of handling avatars

[CdoG]

While I was filing my avatars a couple weeks ago, I was thinking about how much easier it would be if only vBulletin would:

1. Have an option to add the title of the avatar based upon the filename (i.e. if a file was called Final_Fantasy_8_-_Squall.gif, then vBulletin would automatically name it Final Fantasy 8 - Squall)

2. Have another option to be able to put avatars into series (i.e. have the series be FF8, and the avatars be Squall, Rinoa, etc...take a look here (http://squaregamer.com/forum/member.php) for an example of what I mean)

The first option would make my job of giving titles to the avatars much, much, easier, while the second option would allow my forum members to be able to browse through and pick their avatars easier.

Please take these ideas into consideration for either vBulletin 2.1 or some type of hack.

[Vinnie]

I totally agree - I have a zillion users and for each of them to have a different avatar, I need to have about 100 to 200 thousand avatars available. This adds up to a couple of gigs of storage.

However, wading through that many avatars is ridiculous.

Avatars are VERY COOL and I would love to see better support for a huge number of them.

No one wants a duplicate avatar...

[Vinnie]

Speaking of which, does vBulletin allow two users to have the same avatar?

[tubedogg]

100,000 to 200,000 avatars? If you have that many users you will need to be thinking about porting vBulletin to another database that can handle more information, as MySQL will obviously be inadequate.

vB allows two users to have the same avatar.

BTW, are you running vBulletin? I checked your site and I only see DCforums, with 65000+ members but a couple thousand posts???

[Vinnie]

I haven't gone live. The low posts are because threads get archived after 5 days due to performance issues (look in the archives to see all the messages). Even the archives have a 2 month limit because its already at 3 gigabytes of storage.

What do you mean, MySQL won't scale up??? I have it running on a dual P3 1Ghz with 2GB of RAM and two RAID clusters, plus IDE RAID (1 for swap file, one for operating system, and RAID 5 for the database files).

How far will this get me? How many dedicated web servers will I need with PHP for Windows?

The existing DCForum is on its last legs - I had to severely limit the maximum number of preserved posts, as well as turn off a few features. Even having done that, every couple of hours one of the forums index files gets corrupted and I have to rebuild it - all the messages disappear when this happens.

Not only that, the thing is running dog slow. Its painful to even look at it.

This is the new forum:

http://forum.bearsharelabs.com
or
http://208.239.76.100

The URL and IP are temporary until I have completed the deployment, which hopefully will be within the next 24 hours.

I haven't put vBulletin on the other machines in the cluster - has anyone tested this? Am I in for a surprise?? What happens when two users access the same database object at the same time? I didn't see any locking or transaction logic in the source code for vBulletin, does MySQL handle this automatically or something?

Hey while we are on the subject whats the deal with MySQL databases getting corrupted? Is this a disk error? hardware malfunction? problem with MySQL? or referential integrity issues with vBulletin? Please let me know before I open the doors...

[Vinnie]

Don't forget to download BearShare
http://bearshare.com/download.htm

In the program's interface is a "Community" button that takes you to the old forum - this will bring users to the new forum when I've finished the deployment.

[Chris Schreiber]

Originally posted by Vinnie
What do you mean, MySQL won't scale up??? I have it running on a dual P3 1Ghz with 2GB of RAM and two RAID clusters, plus IDE RAID (1 for swap file, one for operating system, and RAID 5 for the database files).

How far will this get me? How many dedicated web servers will I need with PHP for Windows?


Well MySQL will scale up very well... to a certain point. After that you will have problems with table locking during updates which will slow down the database access. Typically, with the machine you have, you could handle 600-700 concurrent users. If you are expecting more, you will need to consider an enterprise database (like Sybase/Oracle), or row-locking tables in MySQL (Gemini/Innobase).

BTW, nice software you have there, I think I'll have to try it out :)

[Vinnie]

Ugh you mean MySQL doesn't have row locking?

Does switching to another database mean changing code?

What are the chances of having it work with MS SQL Server 2000?

Also, what are the chances of running PHP as an ISAPI DLL? Has anyone done it?

When you say 600 to 700 concurrent users is this based on database limitations and not web server limits?


Any details that you can provide would be extremely helpful - I have to assess the scaling issues before I run into them.

To be honest with you I looked through over 50 pieces of software for forums and vBulletin exceeded my expectations.

Do most large forums just have developers working on it full time? Is there no out of box solution?

Will having a large number of forums (places to post) improve the locking issues since the users will be spread out across more disjoint lists of threads?

[tubedogg]

Originally posted by Vinnie

Ugh you mean MySQL doesn't have row locking?
Not stock MySQL, no.

Does switching to another database mean changing code?
It basically means rewriting most of vBulletin.

What are the chances of having it work with MS SQL Server 2000?
Not a snowball's chance in hell unless you rewrite vB.

Also, what are the chances of running PHP as an ISAPI DLL? Has anyone done it?
No but I believe I've been told it will run much slower.

When you say 600 to 700 concurrent users is this based on database limitations and not web server limits?
Database.

Is there no out of box solution?
Not really, at least not in this price range.

Will having a large number of forums (places to post) improve the locking issues since the users will be spread out across more disjoint lists of threads?
No, because it's all going into two tables - thread and post.

[Vinnie]

I looked at InnoDB - does this mean changing the syntax of commands?

What about what Yahoo! does for their e-group...

New thread and posts go into separate tables which are only written to...existing thread and posts come out of where they are now, which is read-only.

Every 5 minutes, an automatic process locks both tables, and transfers the new threads and posts into the regular thread and posts tables then rebuilds the indices.

Disadvantage is that posts can take up to 5 minutes to appear, and there might be a pause while reading threads.

However, the advantage of the bulk insertion are obvious.

-

No good scalable solutions at this price range? I couldn't find any good, owned solution that used a database at ANY price range!

Can you provide some links to really expensive forum software that does what vBulletin does, and scales? I did not find them.

[Chris Schreiber]

Originally posted by Vinnie
Ugh you mean MySQL doesn't have row locking?

No, the default table type in MySQL 3.23 is MyISAM, which is based on table locking. So if you have a very large number of updates, it can cause problems. However there is a new table table, called Innobase, which is based on row locking. I know of at least one vB site which is using Innobase tables and it has elimated locking issues they had with MyISAM tables. Also, in beta, is the Gemini table type being developed by NuSphere, which also has row level locking and can support many more users.


Does switching to another database mean changing code?

Yes, any parts that are queries that interact with the database. Some things are the same, however each DBMS is slightly different in their implementation of SQL. vBulletin 1.1.6 has been ported to Sybase by Matt R at www.fanhome.com


What are the chances of having it work with MS SQL Server 2000?


We ran a poll of our users on which databases they would like to see supported, and MS SQL server did receive a majority of votes. We will be porting to other database systems, however which databases will be first and the timeline have still not yet be discussed.


Also, what are the chances of running PHP as an ISAPI DLL? Has anyone done it?


Yes that is possible, but I believe it may be problematic. Every large vBulletin site (500+ concurrent users and/or 4000+ posts per day) are all running on Linux servers, so this is more reliable and more scalable, IMHO.


When you say 600 to 700 concurrent users is this based on database limitations and not web server limits?


Yes, you can load balance as many web servers as you want (in fact www.bigbigsavings.com as two load balanced web servers and a MySQL database server).


Do most large forums just have developers working on it full time? Is there no out of box solution?


Well I usually help some of the bigger sites with their initial installation and tuning of Apache, PHP, MySQL and vBulletin, but once everything is setup and running properly, there isn't anything you really need to do on a daily basis.


Will having a large number of forums (places to post) improve the locking issues since the users will be spread out across more disjoint lists of threads?

There are certain things as far as vBulletin setup goes, that can help or hurt performance...

[Chris Schreiber]

Originally posted by Vinnie
I looked at InnoDB - does this mean changing the syntax of commands?

Not at all... just compile MySQL to support Innobase, then issue an ALTER TABLE post TYPE=innodb; to convert from MyISAM to Innobase.

[CdoG]

So what does everyone else think of these? Maybe they'll be implemented?

[MattR]

Hard coding the InnoDB and such tables would limit compatibility for people who don't have it; plus the tables are currently in either pre-Alpha stage or Beta stage. It will be a while until MySQL declares them "stable" with MySQL.

However in the future once they are widely available they'll be a part of the standard release of vB, also you can get the version of MySQL in the install and test for the other tables (maybe have it user defined) so that it can be on a per-install basis.

[Vinnie]

Originally posted by MattR
However in the future once they are widely available they'll be a part of the standard release of vB, also you can get the version of MySQL in the install and test for the other tables (maybe have it user defined) so that it can be on a per-install basis.

That sounds awesome