PDA

View Full Version : Database Abstraction Layer


leadZERO
Fri 2nd Aug '02, 11:52am
From looking at the source code it looks like the developers originally intended to abstract the database from the program; and they half did. They wrapped almost all of the database calls, which is a good thing. However, most of the queries are still highly connected with MySQL.

I fear we are already too far down the road in development to have making the program database independent be a simple task. Infact, I imagine it would require a substantial amount of time to completely separate the program from the back-end. Optimally when a new forum is created the admin should be able to just use a drop-in replacement and use the database of his choice (that has a drop-in written of course). Most people probably would not use this since they are already setup with MySQL. However, down the road some forums, including this one, might benefit from using another database engine for speed. Or a site might already be using a certain database engine and not want to have to worry about maintaining two separate databases for users. If they were able to just drop-in a back-end replacement (say a few files) they would have the ability to integrate vBulletin with their existing database without having to do a complete rewrite every time a new version of vB is released.

MySQL is pretty fast as far as databases are concerned, but it doesn't support most of the standard SQL features almost every other SQL engine supports. MySQL 4 is supposed to help bring MySQL closer to a feature set comparable to that of other SQL databases. However, vBulletin is so attached to MySQL 3 making use of these new features in MySQL 4 will be a large task for the development team. And that is why I suggest before we wait any longer we take the steps necessary to make vB3 database independent.

It’s probably too late to consider doing this for the 3.0 release. However, once vB3 starts entering the RC phase it might be wise to have a few people work on this. Luckily it’s the kind of thing that doesn’t have to be completed immediately; both the old non-abstracted and the new abstracted code can coexist as long as MySQL 3 is used. This could be a nice addition to a major revision (vb3.1).

I, personally, am happy with MySQL since all of my servers are UNIX based. However, some of the windows users might like being able to use an enterprise MSSQL server instead. I would however like to use the new additions to MySQL when 4.0 goes into beta. However, as vBulletin stands making use of those new features would be a big job.

Anyone interested in what they have planned for MySQL 4 should check out http://www.mysql.com/products/mysql-4.0/index.html

Some of the highlights include: online hot backup and replication services, ability to change mysqld parameters without taking the server down, FULL TEXT searches to make use of FULLTEXT indicies, full transaction and row-level locking in InnoDB tables, secure links between client and server and many others. MySQL4.1 is slated to include: nested subqueries, stored procedures and multi-table UPDATEs.

Floris
Fri 2nd Aug '02, 11:59am
Since they kind of finalized vB2.x development, I think it is a bit late to indeed go drasticly change a lot of code. And maybe more with vB3.x around the corner, where a lot has been rewritten (guessing that from some threads I read). Said that, in vb3 it might get improved over time. personally I am already happy when it works on my box. And taking under consideration that mysql is almost a default solution for a db on shared/dedicated hosting, going extreme with innodb etc could mean more code and problems. But .. I also like good performance, low loads and proper coding (by the book) to run things smoothly. I have no complaints at the moment for vb2 :)

my 2c

leadZERO
Fri 2nd Aug '02, 4:46pm
Originally posted by xiphoid
going extreme with innodb etc could mean more code

FYI... InnoDB is not a different database engine, or at least what I was referring to... It's a table type in MySQL. http://www.mysql.com/doc/I/n/InnoDB_overview.html

Don't get me wrong, I love vB. And I've never personally encountered a situation when I would have wanted to use any other database other then MySQL. However, I run all of my servers on UNIX and MySQL works well with it. I imagine it takes awhile for Windows binaries to be released for the Windows port; and many of the Windows users might prefer to use the MSSQL engine instead of MySQL.

My main concern is when MySQL 4 is released it will take a lot of time to rewrite many of the queries and the code to let vBulletin take advantage of the new features. If the code was database independent you would just have to modify the drop-in to make use of new things such as transactions or views.

rylin
Fri 2nd Aug '02, 6:17pm
Originally posted by leadZERO
This could be a nice addition to a major revision (vb3.1).

3.11 For Workgroups? ;)

Mike Sullivan
Fri 2nd Aug '02, 6:31pm
Just playing a devil's advocate type here, so don't misinterpret this as a developer saying "ain't gonna happen". I like the idea personally; I just haven't come up with a easily maintainable way to work it yet... (see below)

Well, obviously if we'd do it, targetting just newer incarnations of MySQL wouldn't be the way to go about it. So, lets take the perspective of doing it for Sybase/MSSQL (they're pretty similar) or PgSQL. If you just abstract the queries, you're really not doing much. You need to be able to take advantage of stored procs. How would you consider handling that?

As I'm typing this out, a system like this might work:
$DB->query(QUERY_WHATEVER);
if ($DB->getEngine() == 'MySQL3') {
// code that the RDBMS's with SPs wouldn't need
} Or something similar to that. It'd need to be a bit more complex, depending on the support for SPs in each engine.

I guess the alternative to that would be to stick each query and associated code in a file and then duplicate it for each engine. Of course, that makes bug fixing interesting (gotta fix each bug many times). OTOH, replacing conditionals with polymorphism can be a good thing.

I do like the idea though -- taking advantage of whatever features your RDBMS has instead of working from the LCD. (Hmm... I would like to actually be able to use SELECT COUNT(DISTINCT column)... sometime. :))

leadZERO
Fri 2nd Aug '02, 8:32pm
My thinking was to just take every query and wrap it in a function. If you kept all these functions in one file you could make that the drop in, or even seperate them into a few files for organization purposes. Those wrappers could make use of stored procs if they were available in the given DB or contain the code if there were no stored procs.

Another example could be SQL transactions and MySQL (current) LOCK TABLE. You could make a wrapper addPostToThread() with the necessary parameters. The MSSQL wrapper (or for any SQL-engine supporting transactions) could use a transaction to update the necessary tables (forum, user, thread, post if I remember). However on the MySQL drop-in you could get WRITE LOCKs on the tables (since it doesn't support "transactions").

I think using a lot of wrapper functions would be better then a solution of say making a ton of constants for all the queries. Granted you will be adding a lot more function calls to the stack, but on today's processors I don't think the speed difference would be notable even on a high volume board.

I agree though that unless you come up with a very clever way of doing it maintaining the code could turn into a bitch of a job when you start getting a ton of drop-ins. However, if you DO come up with a solution that pulls the two apart efficeintly I think the user community would pitch in and write their own drop-ins. They would also probably maintain them.

I wasn't meaning just supporting MySQL and it's future versions, I was just trying to use them as an example. And yes, you would have to abstract more then just the queries. I meant not having anything actually querey the engine (even through the class wrapp), take it all out into another level of wrapper functions, ie addUser(), addPostToThread().

Take for example the job of adding a new thread:

In newthread.php or whatever:

errorcheck
permissions check
call addThread( forumid, title, dateline, ... ) in dropin.php returns new threadid
call addPost( threadid, title, text, ... ) in dropin.php returns new postid

The layer for your coding model would be something like what follows:

Database back-end library
\/
Database Class
\/
Back-end Interface Functions
\/
GUI (all vBulletin code)
~
Misc client-side JavaScript


The drop-in replacment for any database would consist of the "Database Class" and "Back-end Interface Functions" layers. I'm not as well versed as I should be on PHP and how much OOP it handles but this would be best implemented as a base and inherited class architecture. As you said, polymorphism can be your friend and as far as I can tell was developed for this specific use.

Well, I started typing this an hour ago and now I can't get my train of thought back, it being Friday and all. So, I'll just leave it at that for now.