Here is the second demo of the asset manager. I was under a hurry so please bear with me. The screencast recorder seems to have a few issues as my audio was cut off in a few places.

I also forgot to mention that you can resize each of the panels by dragging the borders.



http://www.screencast-o-matic.com/watch/cQeZXceqH

I have continued working on the asset management system that I previously demonstrated. The system looks and behaves much differently than what you have seen so far. I have applied a lot of polish to the interface, added a multiple upload system using flash, and added upload progress bars. Managing multiple attachments/assets at once is now possible as well by ctrl-clicking on a thumbnail.

The blog is now working on vB4 as well. I have integrated the new attachment system into Blog 2.1, which is what I am currently calling the version of the blog that supports vB4. The release is not going to have many new features as the point of it is to release a version of the blog that will work on vB4. The integrated attachment system means that attachments in the blog are stored and served by the same scripts that store and serve forum attachments. It also means you can quickly reuse attachments from the blog on the forum, and vice versa.

I will post a new demo of the asset management system on Monday the 23rd or Tuesday the 24th.

Distilling it into it's most basic functionality, the vB CMS allows content authors to create pages of content; categorize those pages into lists; and place widgets of functionality onto those pages to make them more interesting. Although the first release will cover these principals, the CMS is being designed to be far more powerful, which will become more apparent in future development blogs.


Content Types

One of the main goals with the CMS was to allow the creation of 'non static' pages. This could be achieved in a number of ways. From the outset, the simplest way would be to add the ability to create articles that can be listed and managed, in the style of some of the existing content in vBulletin. However, we would eventually need to add other types of content.

As an example, it's easy to compare social group discussions with threads and wonder why they aren't pretty much the same thing. Unfortunately the code in vB3 for these (and other types of content) doesn't lend itself well to reuse. Because of this, the code for things such as inline moderation, read marking and subscriptions of these two similar features is currently duplicated and modified to suit each as if they were almost entirely separate. As vB3 has grown in functionality, the benefit of consolidating these separated features has grown too.

The CMS is designed to be content agnostic, allowing all manner of content types in a way that allows us to create and implement features that can apply to all content equally; without the need to duplicate the effort for each supported type of content. For example, if I wanted to create a user rating feature; it should be available for all content regardless of whether it is an article, gallery image or video.

Conversely, it should also be much easier to create a new type of content and allow it to be used for pages, while automatically benefiting from all of the existing content orientated features without having to write lots of specific code just for the new type.

Not only will the CMS allow the vBulletin developers to quickly create new content types for your pages, it should also be very exciting for mod developers to come up with their own content types and widgets.


Widgets

Along with the main content, the vB CMS also allows you to place widgets on the pages. Widgets are used to express non page content and allow you to liven up pages with related content, provide further navigation options to the user, to highlight other site content, or do pretty much whatever widget developers can think of. Widgets are a very simple concept and are fairly standard to content management systems, but they should not be underestimated in terms of the features that they can provide.


What's Working Now

In the future I'll be talking about some of the more technical aspects of the CMS and the framework that it's based on, for now I'll walk through some of the functionality that's currently up and running, and discuss features that should be included in the initial release. However, there are a couple of caveats -

Firstly it's important to bear in mind that most of the work done so far has been on the framework, and abstract features that will enable cool things to happen in the future. I contemplated discussing some of these concepts and the design goals behind them in this blog but decided against it. Not only can abstract concepts be overly technical, but describing them requires me to allude to examples and potentials that may never actually be developed. I'd rather discuss concrete features in future blogs, but will be happy to go into more detail in the blog comments.

Secondly, as most of you know, vBulletin 4.0 will be shipping with a new style. As that isn't finished yet, the CMS isn't using it - but as it will be using it eventually I haven't wasted much effort in making it look pretty.


Creating a Page

The design of CMS pages and the widgets that are included on them is defined by the Layout that the page uses. Freddie published a screen cast about Grid and Layout Management last month that goes into a lot of detail. Essentially, you need to create a layout before you can create a page; although the CMS should ship with a few default layouts.

After the CMS is installed and running, a default page will be available. Here's a development test page:



From here you can enter Edit Mode by clicking Edit this Page. In the future, there will be a nice toolbar UI for all of the page management tools, for now it's just a bunch of links.



Once in edit mode, you can now edit the content on the page; configure the content; configure any widgets; add content or delete the page.

Here you can see a text area for editing static html in a development content type. There is also an edit link in the toolbar for the content, next to it's config link. Depending on their individual behaviour, widgets and content may be edited inline while the page is in edit mode; or be edited with a dedicated UI by clicking the edit link on the widget or content. The static html content type here has the edit link enabled for demo purposes only, as this particular content type is edited inline.

Content and widgets may also allow you to configure how they behave on the page. Clicking the config link in the widget toolbar opens the config UI.



In this instance, clicking Config on a widget's toolbar while editing a page will allow you to configure the widget for that specific page. However, widgets can also have a global config that can not only determine global settings, but also define what content authors can change per page. In a future blog we will take a closer look at how widgets are created and configured.

Clicking Add Content allows you to create a new page.



When you create a new page you can select the type of content to use as well as the Style and Layout. As mentioned before, the Layout is predefined and determines the layout and widgets that will be included on the page.

Optionally you could add the title to the URL or allow it to be generated automatically.

The location defaults to the page that you were viewing when you clicked 'Add Content'. It specifies where the new page will be on the site as a whole. There will be a better UI to browse the site structure (a la Freddie's asset management), instead of a simple drop down.



You can of course delete a page. After a confirmation, the page will be removed and you will be redirected to the nearest parent content.


The walkthrough above appears simple, but is just enough for basic content driven websites. A lot of effort is being put into the code behind the CMS to ensure that it is flexible enough for many powerful features to evolve in the future.

It's possible to create subsites with different styles and layouts; widgets can read the content on the page for more advanced and dynamic features such as content highlighting or automatic related content links. Mod developers can develop content types and widgets in their own packages with minimal hooking required.

As I have mentioned previously, a big change in vBulletin 4 involves the attachment system. I am currently working on a system that allows you to quickly reuse attachments through vBulletin and vBulletin products.

.. and so I have done a run through of the current state of the system for you to view.



http://www.screencast-o-matic.com/watch/cQn1o7nPG

If you didn't notice, this uses javascript, a lot of javascript, so if you don't have javascript enabled, then you will fall back to the familiar vB3 upload form.

A major feature slated to debut with vBulletin 4.0 is a new front-end style marked-up with semantic XHTML and styled with CSS.
The task of producing the new templates using this scheme has thrown up all manner of questions, challenges and show-stopping problems, one of which I will discuss here.

Background

In vBulletin 3.x, the primary markup vehicle was the HTML <table> tag and almost all elements on vBulletin pages could be summarized as having the following basic structure:

Using this scheme, there is semantically no more value to the main heading than to any other tag in the block. Furthermore, the <table> tag and its contents strictly dictate how the element will appear; CSS has only a limited scope to change the fundamental structure of <table> elements.

It’s worth noting at this point that the use of the <table> tag for this (or any other) purpose does not mean that the markup is not valid XHTML. In fact, the majority of pages in vBulletin 3.x validate against the standard to which they are built, namely XHTML 1.0 Transitional.

However, it’s clear that a major part of a move to semantic XHTML will involve using more suitable markup for page elements, making use of tags that more accurately describe their content and contextualize it on the page.

Threadbit

One page that presents quite a conundrum is forumdisplay.php: the page that shows the list of threads in a forum. Let’s take a look at how the thread list has been presented in past versions.

Firstly, we have vBulletin 1.x.



Here, the thread data is presented in classic tabular fashion, with a single data point in each column, with the exception of the ‘Last Post’ column, where we show both the time and the writer of the most recent post to the thread in question.

This is how the thread list looked in vBulletin 2.x.



The data is still largely tabular, though we are starting to see content other than just the title in the ‘Thread’ column, namely the icon to represent a thread containing attachments and the ‘Poll:’ prefix.

With vBulletin 3.x, there was an attempt to cut down the number of columns.



We can see that the name of the thread starter has moved into the same column as the thread title, placed on a new line. The rating stars have also migrated into the same column and there are several more icons representing different attributes of the thread.

The question is:
Is the vBulletin 3 thread list still tabular data?

At first sight, yes it is. We have a number of columns each containing data and the arrangement looks like a grid. But do we consider the thread list to be tabular data simply because it is arranged in this way?

If we extract all the data and put each individual item into its own column, as should arguably be the case in order for the use of <table> to be semantically correct, we would end up with the following columns:

1. Status icon (read/unread, hot, moved, locked, posted-by-viewer)
2. Post icon
3. Prefix
4. Title
5. Thread poster
6. Replies
7. Views
8. Rating
9. Last post date / time
10. Last poster
11. Poll indicator
12. Sticky indicator
13. Attachment indicator
14. Tag indicator
15. Subscription indicator
16. Deleted thread indicator
17. Contains deleted posts indicator

A table with this many columns is not going to win any design awards. It would be fine to view in Excel, but as an immediately accessible, visually attractive web page, it’s going to look pretty terrible.

The Requirements

The basic structure I require visually is that of a table. I need a liquid column (one that stretches to occupy available space) followed by a number of fixed-width columns. Each column must be capable of supporting a background colour or image that will fill the cells.

The old way to achieve this would be to use a <table> tag with rows styled in the following way:Of course, we want to avoid using a <table> tag if at all possible, given that our thread list is not strictly tabular data. Happily, most modern browsers give us an alternative that offers all the benefits of a <table> but without the semantic ambiguity, namely the CSS attribute display:table-cell. This would allow us to produce a layout visually identical to a table, but using semantic mark-up, such as this:This screengrab shows how Firefox 3 renders the two different HTML snippets. It’s obvious that they are visually identical.



This markup allows radical restyling of the appearance of the thread listing and completely avoids the problematic <table> element. So, problem solved? Unfortunately not. There is one teeny tiny problem that is getting in the way of our goal. It’s called Microsoft Internet Explorer.

Making IE Work

Although support for display:table-cell has been introduced in Internet Explorer 8, the reality of the situation is that IE8 is not yet out of beta, leaving IE7 as the current primary version. What’s more, a significant portion of the web-browsing population is still using (shudder) IE6. Prior to IE8, display:table-cell is not supported at all in Internet Explorer. Given that Internet Explorer’s market share still represents a significant majority, there must be a workaround so that Internet Explorer users are not presented with a disastrous mess when they visit sites running vBulletin.

If we feed IE7 with the same CSS as we present to other browsers, IE7 will ignore the display:table-cell instruction, resulting in the cells being represented as horizontal blocks.



Inline-Block

Internet Explorer 7 does implement a CSS property that can offer some help in this instance, namely display:inline-block. We don’t want other browsers to see the inline-block instruction, so we will create a new stylesheet surrounded by Internet Explorer conditional comments.

Unfortunately, IE7 seems to have some bugs related to inline-block that need to be carefully prodded to make the system work. For some reason, a simple declaration of display:inline-block seems to have no effect, but it can be made to work by following it with a display:inline call. Yes, it’s that weird.

We can see from the preceding screengrab that IE is now attempting to place the blocks next to each other rather than stacking them vertically, but the liquid column is expanding to fill the whole width of the container, rather than only occupying the space left by the fixed columns. For the time being, we’ll fix this by specifying a fixed width for the liquid column. We’ll come back to the liquid problem later.



Finally, IE7 is displaying all the blocks horizontally in table fashion, but unlike a true table, the cells are not all of equal height.

Equal Height Cells

We start by specifying vertical-align:top for all cells, which will cause them to be aligned along a single top edge rather than the bottom edge.

The next task is to make all the cells of equal height – all must expand or contract automatically according to the natural height of the highest cell in the row.

We start by adding a massive amount of padding (9999px) to the bottom of all cells, such that it will extend well past the bottom of the browser window. We then reign that back in with a negative value (-9999px) for the bottom margin. This will look extremely strange and drop all the cells of the bottom of the page, until we instruct the row to hide any overflowed content.

Single Liquid Cell

We now have (what looks like) cells of equal sizes, which expand and contract vertically as they should. The last thing that remains is to get our liquid column working.

Internet Explorer supports expressions in CSS, which is a very powerful system allowing CSS values to be generated by Javascript. We will make use of this system to calculate the amount of space left in the row container after the fixed width cells have been inserted. The remaining width will be applied to the liquid column. I wrote a little function called liquid_td to handle this.

No-Javascript Failsafe

Everything now looks correct, though our use of expression falls over if we encounter a visitor running Internet Explorer with Javascript disabled.

To handle this worst-case scenario, we must abandon our liquid-fixed-fixed layout and switch to a completely liquid system, specifying all the widths in percentages. This will require a further stylesheet, this time surrounded by <noscript> tags to hide it from browsers that don’t need it.

Unfortunately, this flings us headlong into Internet Explorer’s lack of common sense when it comes to math calculations. One might expect that 60% + 15% + 25% = 100%, but Internet Explorer disagrees from time to time. The problem appears to be one of unintelligent rounding. Here are a pair of examples that illustrate the problem.

Example 1: Container width = 400px.

• 60% of 400 = 240
• 15% of 400 = 60
• 25% of 400 = 100
• 240 + 60 + 100 = 400.
• Result: all columns fit.

Example 2: Container width = 399px.

• 60% of 399 = 239.4, rounds to 240
• 15% of 399 = 59.85, rounds to 60
• 25% of 399 = 99.75, rounds to 100
• 240 + 60 + 100 =400
• Result: last column drops because 400 pixels do not fit into 399 pixels.

To deal with this problem, we need to set the width of the first cell to 0.1% less than the desired value. This appears to instruct IE to round down instead of up, resulting in the correct number of pixels being calculated.

Conclusion and Discussion

Our final system works like this:So, now we have a working system to emulate table cell behavior in our target browsers.



The question now is:
Was it all worth it, just to avoid using a <table> tag?

One of the most common complaints about vBulletin's upgrade process is applying the template changes. While the template history feature helped a little bit, it didn't go far enough. vBulletin 4.0 will introduce a feature to make applying template changes much easier: automatic merging and 3-way comparison.

Whenever you upgrade, we will automatically look for templates that have changed and see if you customized those templates. If you have, we will automatically try to merge the changes in. If this succeeds, you won't have anything else to do (in most cases). If it fails, your customized template won't be updated--like in vBulletin 3--and you'll have to apply the changes yourself. However, that process is improved as well. More on that later.

Roughly, the merging process goes like this:

• Find the differences between the old default and the new default. ("old-new")
• Find the differences between the old default and your customized version. ("old-custom")
• Start walking through the template. If we find a changed spot in old-new but it's not changed in old-custom, use the old-new version. If we find a change in old-custom but not old-new, use the old-custom version. If neither changed, use either. If both changed, that's a conflict and we can't do the merge.

If you're interested in more technical details, check out this page or this university paper.

Most of this happens behind the scenes. However, you'll see some changes to the "Find Update Templates" screen:


The "View Highlighted Changes" link will take you to this page:

This lets you view the 3-way merge results visually. The yellow lines indicate areas changed by merging, while the red blocks indicate conflicts. The conflicting areas show you the values in all 3 versions of the template (old, new, customized) to let you choose how to resolve it.

Of course, trying to do the edit from this page would be a challenge, so if you click the "Edit Merged Text" link, you'll be taken to a normal looking template editor:

However, the text that you're editing is slightly different from the norm. Any unchanged or merged areas are as you'd expect, but the output from a conflict is very different. All 3 possible values for a conflict will be shown, so you can resolve it correct. And don't worry, if you try to save it while there's still a conflict, it will warn you.

I should note that the automatic merging is pretty conservative. "Adjacent updates" (when a customized version adds a line immediately after a line that was changed) will trigger a conflict. In some cases, it'd be ok for this merge to go through, but there are other situations where things would break if the change were applied. Regardless, the new conflict management system should allow you to resolve the problem more quickly.

Hopefully this feature will make upgrading a customized vBulletin easier!

I am still on target to release Blog 2.0.1 next week. You may notice quite a few bugs in the bug tracker, but none of them look serious and so I hope to knock them out in short order.

As for the vB4 Attachment Management demo, I am hoping I have something to show you next week but to be realistic, it won't be ready until the week of Feb 2nd.

I will give you a screenshot of its current state, as long as you realize that it is still in development and missing controls.

vBulletin 4.0 will include a few features that will help your community content be better indexed by search engines.

Search engine optimisation is an inexact science, and when putting together the slate of features we spent time researching the latest thinking around what factors lead to positive benefits, including absorbing existing research on sites such as seomoz.org.

The features that have been created for vBulletin 4.0 are the main factors that we consider will have the greatest positive impact in terms of search engine ranking. Instead of including hundreds of options that can be infinitely tweaked, the features chosen consist of semantic HTML packed with content meta-data, the reduction in the amount of duplicate content and search engine friendly URLS for forums, threads and member profile pages.

Built in features

Keyword Use in Title Tag - The HTML title of the key vBulletin pages now better represents the content within the page, and extraneous information (such as "Page x") have now been removed.

NOFOLLOW tag for all external links.

Duplicate content reduced - by altering the operation of the showpost.php file to redirect cleanly to showthread.php.

Meta description and keywords - now populated with relevant, meaningful page level content, including tag data and content snippets.

Friendly URL options

For 4.0, there is a new area within the admin control panel that will allow you to switch on friendly URLs that will work in conjunction with server redirects (.htaccess) where supported.



For threads, the following URL structures for threads, member profiles and forums are created for each of the options:

Standard URLs - as exists at present:

• showthread.php?t=12345
• member.php?u=12345
• forumdisplay.php?f=123

Basic Friendly URLs:

• showthread.php?1234-Thread-Title
• member.php?1234-Username
• forumdisplay.php?12-Forum-Title

Advanced Friendly URLs

• showthread.php/1234-Thread-Title
• member.php/1234-Username
• forumdisplay.php/12-Forum-Title

Mod Rewrite Friendly URLs

• threads/1234-Thread-Title
• members/1234-Username
• forums/1234-Forum-Title

Search Engine Sitemap

vBulletin 4.0 will include a built in XML site map system that provides a machine readable index of forum posts and threads.

Google and other supported search engines will be able to access the sitemap, which will facilitate a greater amount of site content being included within the search engine index.


If you have any questions, suggestions or thoughts on the planned SEO functionality for vBulletin 4.0, please post a comment. Thanks!

More development blog posts coming soon.

-James

Greetings all,

For those of you whom I haven't yet met by phone, chat or email, I would like to make a brief introduction and respond to some of this threads lively discussion.

My name is Ray Morgan, and I joined Internet Brands in September (minus a recent planned hiatus) to work full-time with Jelsoft on the development of the 4.x series of vBulletin. My time is devoted entirely to vBulletin, and I'm commuting every few weeks between Jelsoft's office in Pangbourne, UK and Internet Brands in Los Angeles.

For those who were curious about my background, I've been developing commercial software for more than twenty years and worked exclusively in the internet space for the last ten, most of those on large-scale commercial sites. My role in connection with vBulletin is a mix of product management, process engineering, project management and architecture.

A few of you have raised some valid questions about how (and why) the approach to vBulletin development has recently changed, and what the implications are. I'll try to clarify below.

Will we rewrite vBulletin?
Yes, we are rewriting vBulletin. The current code base has served us well, but it's now time to rebuild it.

This task is enormous. The current incarnation of vBulletin has roughly a quarter of a million lines of code and took many tens of thousands of developer-hours to produce. Re-implementing its rich feature set, complex business logic, and years of bug fixes and tuning is a huge undertaking. The upcoming 4.0 release represents the beginning of that process.

While, as James and Kier stated, 4.0 will not be a complete rewrite, the 4.x series will be, and the first wave of refactoring to ship with 4.0 will cover aspects of multi-content search, attachment management, style management, presentation design and more. Additional components will be refactored with each subsequent release.

Additionally, the 4.0 release will introduce some major new customer-requested features, like CMS, ad management, and some key SEO features. For those who use highly customized templates, there is also a greatly enhanced utility for diffing and merging them to make future upgrades much easier than theyve been in the past.

Why will vBulletin be rewritten incrementally instead of all at once?

With a team of eight engineers, a complete rewrite is no less than an 18 to 24 month project, and that assumes that no new features are added in the process. This is a competitive market, and two years is a very long time to go without new feature releases.

There are two primary reasons why we have chosen to approach the rewrite incrementally rather than as a big-bang, all-at-once proposition:

• Responsiveness to community feedback - We base our product development decisions on the suggestions, critiques and debates on this forum. We've heard loud and clear that (almost) nobody is willing to wait years for improvements to be rolled out. Moreover, the more frequently vBulletin is released, the sooner we hear your feedback, and the faster we can respond to it.
• Predictability. We hear loud and clear that vBulletin users want a predictable release cycle you can depend on. While in theory a clean-slate rebuild sounds good, the reality is that large-scale software rewrites are mine fields that are notoriously difficult to plan and execute.
  Breaking the work into smaller, more frequent releases makes our estimates more accurate and ultimately yields a higher quality product. It also somewhat lessens the risk that the new architecture will hand us ugly surprises, such as new performance issues we hadn't seen before. (A very real problem, since such surprises nearly always surface in rearchitecture projects.)
  
  And yes, even radical rearchitecture can be done incrementally.

The main differences are that:

• there will be incremental releases along the way, rather users having to wait the full two years to enjoy the benefits of our work
• what was previously called 4.0 now is now more accurately the 4 series

The overall time frame for the rewrite is about the same as previously expected, probably on the order of two years, as James stated previously.

While we know that we can't please 100% of users 100% of the time, Jelsoft and Internet Brands are deeply committed to providing vBulletin users a product they will love, and every decision we make is driven by that mission.

I have recorded a Screencast of a quick run through of the current grid and layout management systems of the CMS. The two areas are simply building blocks to creating a page. A page consumes a layout and a layout consumes a grid.

These areas are the only items that I have worked on for the CMS so my comments are going to be limited to this subject.


http://www.screencast-o-matic.com/watch/cQVn6inpa

Remember, these are just components of building a page and I have not included the method by which a page is created. I have not included the method by which widgets are configured.

Looks like simsim, nailed it really quick with the CMS guess as to the surprise that I alluded to in my last blog entry.

CMS development is very much a group effort. I have been doing more of the javascript related components. I hope to be able to show you some of the interface for moving content blocks about on the layout soon.

I've also implemented a generic overlay interface, the "lightbox" method that attachments use, that we will be utilizing in the CMS to accept some user input. Below is an example of the current state of the overlay, which is subject to change.



Yes, I know it isn't much...

vBulletin 4 development is also a group effort.

I have implemented a consistent interface for defining friendly URLs in the code base as well as in the templates.

The attachment system has been extended so that products and vBulletin now share the same core database tables and filesystem for reading and writing attachments. Reuse of existing attachments is now automatic when a duplicate file is detected on upload. The current attachment interface will be replaced with a javascript based interface that gives an overview of your uploaded content, allowing for easy insertion of existing content. The interface is planned for the first vB4 release but I can't promise that it will make it.

After vB 4 is released, the blog will have a release that imports the blog attachments into the centralized attachment system.