The Life of Charlie

Russian Dolls and X-ray Specs

2006-09-16T17:28:00.000-07:00

In an earlier entry titled ‘The Cache is a Shadow, Not a Box,’ I discussed how an incorrect mental image played havoc with Charlie. It may come as no surprise that another incorrect mental image has caused yet more problems.

The mental image I have of object-oriented inheritance is that of Russian Dolls. Here is a little bit of code to provide an example of how I see inheritance working:

public class MediumDoll
{
    private Object SmallDoll;
}

public class LargeDoll : MediumDoll
{
}

To put the code into words, I see the private SmallDoll as being inside its containing MediumDoll. If LargeDoll inherits MediumDoll, then I see the SmallDoll as being deep within LargeDoll. Because SmallDoll is of private scope, LargeDoll cannot access it directly. But even if LargeDoll cannot access the SmallDoll, the SmallDoll is still inside the LargeDoll—but with the MediumDoll separating the two.

If you know the basics of reflection in .NET, you will know that it allows us to both see and use members that are otherwise hidden from view. Without reflection, SmallDoll would be completely hidden from any code outside of MediumDoll. But with reflection, we can see and use SmallDoll even from outside of MediumDoll. A completely unrelated class, such as ToyMaker, could employ reflection to see and use the SmallDoll that is otherwise hidden inside MediumDoll.

Because reflection gives our code x-ray specs, I believed that if I pointed reflection at the LargeDoll, I could see through to the SmallDoll. This is not the case.

Putting the analogy aside for a moment, what I was trying to do was rectify a weakness in Charlie’s Entity System. In writing Charlie’s First Code, I gave each entity an ID value, and a public method that changed that ID value.

namespace Charlie.Framework.Business
{
    public class Entity
    {
  
        public Int32 Id
        {
            get
            {
                return this.id;
            }
        }
        private Int32 id = -1;

        //  
        //  While this method has the same scope (public) of the Id
        //  property, we'll use a separate method to set the Id, as
        //  the Id property is properly meant to be a read-only
        //  property from the point of view of all consuming code.
        //  While we could use reflection to "hide" this SetId method
        //  as private, that seems like needless complexity.
        //  
        public void SetID(Int32 id)
        {
            this.id = id;
        }
              
    }
}

After an entity is retrieved from the database, its ID value should not be changed, which is why its ID property is read-only. But as an entity is being retrieved from the database, there needs to be some way to set that ID value. For this reason, I kept the ID property as read-only, but gave the class a separate SetID method. In the notes I admitted that this was a hack to avoid the complexity of using reflection to set an otherwise read-only value.

Inspired by the success of using reflection to inspect Charlie’s Bucket Full of Plugins, I decided to finally get rid of that little SetID hack.

Here then is a concrete example of the Russian Doll analogy:

public class Entity
{
    private Int32 id;
}

public class Article : Entity
{
}

What I needed to do was update Charlie so that if it was given an Article that had been retrieved from the database, it could set the private id field. The id field is inside the Entity class, from which the Article class derives. To repeat, I believed that reflection would give Charlie the x-ray specs needed to see through the Article to the private id field hidden within. Here is a cut-down version of the code I tried:

FieldInfo field = article.GetType().GetField("id");
if (field != null)
    field.SetValue(entity, 3);

The code says, “For this article, grab the field named ‘id’, and set its value to 3.’ This did not work.

I firmly believed that if given an Article (or a LargeDoll), reflection should allow me to see through to the private id field (or a SmallDoll). My persistent mental image of reflection as x-ray specs meant that all of my debugging efforts were directed at the x-ray specs. There are many ways to fine-tune the GetField method shown above, including a promising-looking FlattenHierarchy option. Yet every attempt failed.

After failing continuously to set the private id field, I realized that my concept of reflection as x-ray specs must be wrong. (That seems obvious in retrospect, but we are typically unaware of the mental images that we are using.) So once I became aware that I was using a mental image of x-ray specs, I put it to one side and looked instead at the Russian dolls. Specifically, I changed the code to “open” the dolls until it got down to the tiny doll within, at which point the reflection would work to peek inside. Here is a simplified version of the code that did work.

Type type = article.GetType();
while (type != typeof(Entity))
{
    type = type.BaseType;
}
FieldInfo field = type.GetField("id");
if (field != null)
    field.SetValue(article, 3);

The emboldened code keeps opening a derived class until it gets down to the base Entity class (keeps opening dolls until it gets down to the MediumDoll), at which point the reflection can see inside and change a private member.

I am a little loathe to publish this weblog entry, as I feel sure that I am missing something. In reading about reflection, I have never before seen a discussion about unravelling derived classes in order to see and use private members. Still, this weblog represents an honest account of Charlie’s development, and this does represent another little step.

by Alister Jones |

A Bucket Full of Plugins

2006-09-16T15:48:00.000-07:00

In the previous weblog entry I said that Charlie included a hack where the name of each plugin was hard-coded. The example given was as follows:

// Security Plugin
ISecurityPlugin securityPlugin =
    (ISecurityPlugin)pluginFactory.InstantiatePlugin(
        "Charlie.Security.SecurityPlugin, Charlie.Security");

In the above code, the string parameter consists of two parts. The first part of ‘Charlie.Security.SecurityPlugin’ is the name of the class to load up, and the second part of ‘Charlie.Security’ is the name of the assembly in which this class can be found. With these two bits of information, Charlie can create an instance of the Security plugin.

An easy solution to removing the hack would be to relocate this string information to the Web.config file. In that way, the required string value is no longer hard-coded. But there is a problem with such a simple solution. The problem is that the Web project must still have intimate knowledge of how all of the plugins are structured. The Web project must know both the names of plugin classes and the names of the assemblies. This somewhat defeats the purpose of plugins, which is that Charlie can operate without intimate knowledge of its plugins.

I decided that each plugin would be given a friendly name (such as ‘Charlie.Security’), and each would be referenced by its friendly name. This removes the need for plugins to be referenced by class names and assembly names. In turn this allows plugins can change their structure without affecting Charlie’s ability to use them.

I started by creating a new PluginInfo attribute which, when applied, looks like this:

namespace Charlie.Security
{
    [PluginInfo("Charlie.Security")]
    public class SecurityPlugin : ISecurityPlugin, IPlugin
    {
    }
}

That’s all well and good, but how can Charlie find plugins when all it has is a friendly name, and does not have the assembly name or class name? I applied a technique I had seen previously in Sprocket. When Charlie first loads up, it inspects all the assemblies in the \bin folder, and picks out any classes that implement the IPlugin interface. Charlie then maintains a record of each plugin and the friendly name associated with that plugin. Now if Charlie is asked to use the ‘Custom.Security’ plugin, it knows which plugin goes by that friendly name. In other words, Charlie is no longer told the assembly names and class names of plugins to use, but now finds this information for itself. The cut-down version of the code looks like this:

foreach (Assembly assembly in AppDomain.CurrentDomain.GetAssemblies())
{
    foreach (Type type in assembly.GetTypes())
    {
        if (type.GetInterface("IPlugin") != null)
        {
            ConstructorInfo constructor = type.GetConstructor(new Type[0]);
            Object[] attributes = 
                type.GetCustomAttributes(typeof(PluginInfo), true);
            PluginInfo info = null;
            if (attributes != null && attributes.Length > 0)
            {
                info = (PluginInfo)attributes[0];
            }
            String name = (info != null) ? info.Name : String.Empty;
            /*
             *  At this point we know this type is a plugin, we know
             *  its friendly name, and we have its constructor.
             *  Charlie now stores the name and constructor as a
             *  key/value pair. This means that Charlie can create
             *  an instance of the plugin whenever it is passed its 
             *  friendly name.
             */
        }
    }
}

I then updated the Web.config file so that it passed in the friendly name of each plugin to use. Here is an example:

<add setting="DefaultSecurityPlugin" value="Charlie.Security" />

So what have we achieved here? Well, previously Charlie had to be told the name of the class to be used as its Security plugin, and the name of the assembly in which to find this class. If the class name or assembly name ever changed, Charlie would stop working. Now, Charlie only needs to know the friendly name of the Security plugin, and behind the scenes the plugin can change its name or assembly.

I then decided to push it one final step further. I noticed that all of the default plugins were Charlie plugins. A particular web site can specify another security plugin, such as ‘Customer.Security’, in the absence of which Charlie will default to its own plugin, which is ‘Charlie.Security’. I throught to myself, “Charlie knows the friendly names of its own default plugins—it does not have to be told these names.” So I removed the Web.config entries that specified the default plugins, and instead left it to Charlie to recognise one of its own. A particular web site can still specify an alternative, but otherwise Charlie uses its own default plugin.

So the hack has been removed. When it starts up, Charlie inspects the \bin folder and keeps a record of any plugins it finds. Charlie also recognises one of its own plugins, which it will use if a particular web site does not specify the use of an alternative plugin. Charlie no longer needs to be given a class name, assembly name, or friendly name. This is the sort of self-configuration that I am now trying to get Charlie to achieve. So far, so good.

by Alister Jones | Next up: Russian Dolls and X-ray Specs →

Charlie’s Own Two Feet

2006-09-15T17:58:00.000-07:00

If you have been following my weblog for a while, you will know that it has been nearly three months since I last created an entry. The reason is that shortly after the last entry I accepted an invitation from Telligent to work on the www.asp.net website. In order to meet my new responsibilities, I needed to put Charlie to one side for a while. However I miss working on Charlie, and I would like to keep the project ticking along. And so we return to the story of Charlie.

Right now Charlie includes two hacks and suffers one nuisance. I have decided to address these issues as a way of getting back in to Charlie.

The first hack is that the names of the plugin assemblies are hard-coded. Here is an example:

// Security Plugin
ISecurityPlugin securityPlugin =
   (ISecurityPlugin) pluginFactory.InstantiatePlugin(
      "Charlie.Security.SecurityPlugin, Charlie.Security");

The second hack is that while Charlie is designed to serve multiple websites from a single installation, a fudge exists to “feed” Charlie a single site ID value. Charlie should properly figure out the ID from the URL of the incoming request.

The one nuisance is that any change to Charlie’s database schema or database content requires making the change in two places. To start with I fire up Enterprise Manager and make the change (since Charlie currently has no administration screens). I then need to duplicate the change in the SQL installation scripts I maintain for the occasions in which I need a completely fresh database installation. This duplication of work has proven to be error-prone. I would like to be able to make the changes just in the installation scripts, and then force Charlie to perform a fresh database installation.

If I address these issues, Charlie will largely be able to self-configure. So let’s see if I can encourage Charlie to stand on its own two feet.

by Alister Jones | Next up: A Bucket Full of Plugins →

Implementing Raw Templating - Part 2

2006-06-17T15:36:00.000-07:00

In the previous weblog entry, I introduced the IWebTemplateParser interface, which looks like this:

public interface IWebTemplateParser
{
    String[] ControlDirectives { get; }
    String Parse(String template);
}

The first line allows each plugin to “tell” Charlie about any custom server controls that the plugin uses. But what is the purpose of the second line? Well, here is the next template that I inserted into the database:

<html>
    <head>
    </head>
    <body>
        <h1><insert:Text name="Welcome" /></h1>
    </body>
</html>

This is an example of the declarative templating system that I am now implementing. There are two things to note about the bold tag shown above.

First, using <insert:Text /> and <insert:Snippet /> and <insert:Image /> would require that each of these custom controls reside within the same assembly, since each prefix (being ‘insert’ in this case) can point only to a single assembly. However, while I want the website owner to have only one prefix to learn, I want each control to point to a different plugin and, therefore, a different assembly. So <insert:Text /> should go to the Globalization plugin, since that is where the text resources are handled. And <insert:Snippet /> should go to the HTML plugin, since that is where HTML snippets are handled. So I need some way of allowing the same prefix of ‘insert’ to point to different assemblies.

The second thing to notice about the above tag is that it contains no runat="server" parameter. That is a functional requirement of ASP.NET that should not be forced upon the website owner.

So, it is to solve the above two problems that the IWebTemplateParser has the Parse method. Before Charlie uses the template it draws out of the database, it will give the template to each plugin. Each plugin can then “fix” the template before it goes to the Page.ParseControl method. The Globalization plugin, for example, will look for the following tag:

<insert:Text name="Welcome" />

If found, the tag will be changed to this:

<globalization:Text name="Welcome" runat="server" />

With that simple replace operation, we have solved the two problems. The general ‘insert’ prefix has been replaced by a specific ‘globalization’ prefix. (Remember, the first line of the IWebTemplateParser interface is where each plugin can “tell” Charlie about custom prefixes.) We have also added the runat="server" attribute that ASP.NET requires.

Here is the full code from the Globalization plugin:

namespace Charlie.Globalization.Interface
{
   class TemplateParser : IWebTemplateParser
   {
      public String[] ControlDirectives
      {
         get
         {
            String[] directives = new String[1];
            directives[0] = 
             @"<%@ Register TagPrefix=""globalization""
                   Namespace=""Charlie.Globalization.Interface""
                   Assembly=""Charlie.Globalization"" %>";
            return directives;
         }
      }

      public String Parse(String template)
      {
         String from = @"<insert:Text(\s)";
         String to = @"<globalization:Text$1runat=""server"" ";
         template = Regex.Replace(template, from, to);
         return template;
      }
   }
}

Right now, the regular expression-based find-and-replace operation within the Parse method is overly simplistic. But right now I am just making sure the functionality works—the find-and-replace operations can be tweaked later. But the point is that the above code is very simple. Each plugin can inspect the template for known tags, such as <insert:Text>, and make any changes it requires. It will make changes so that a custom server control will be called into play. In this example, the Globalization plugin will use a custom server control that is an extension of the Literal control. Here is its code:

namespace Charlie.Globalization.Interface
{
   public class Text : Literal
   {
      public String Name
      {
         get
         {
            return this.name;
         }
         set
         {
            this.name = value;
         }
      }
      private String name;

      protected override void OnPreRender(EventArgs e)
      {
         this.Text = ResourceManager.Current.GetString(this.Name);
         base.OnPreRender(e);
      }
   }
}

This too is about as simple as a custom server control can get. So, what have we achieved here? Well, previously a template using localized text needed to be a User Control that included both a declarative server control and procedural code:

<%@ Control Language="C#" 
    Inherits="Charlie.Framework.Interface.TemplateControl" %>
<%@ Register 
    TagPrefix="aspx"
    Namespace="Charlie.Framework.Interface.Controls"
    Assembly="Charlie.Framework" %>
    
<h1><aspx:Label ID="Welcome" runat="server" /></h1>

<script runat="server">
    private void Page_Load(Object sender, EventArgs e)
    {
        Welcome.Text = ResourceManager.Current.GetString("Welcome");
    }
</script>

Now, the relative complexity above has been replaced with a very simple declarative tag:

<h1><insert:Text name="Welcome" /></h1>

That is all the code that is needed to insert localized text into the template. If the English version of the page is requested, the heading will be “Welcome”. If the French version of the page is requested, the heading will be “Bienvenue”. This is the sort of simplicity that I want from the “raw” templating system that I have devised. And so far, so good.

by Alister Jones | Next up: Charlie’s Own Two Feet →

Implementing Raw Templating - Part 1

2006-06-17T14:30:00.000-07:00

Over the last few weblog entries, I have described the “raw” templating system that I will add to Charlie.

In describing it as a flexible templating system, I noted that I would use a default-and-specific approach to templating. A given type of webpage (such as an Article or a Photo Gallery) would have a default template, but a specific page can override that default template and effectively “tweak” it for just that specific page. This is the same cascading logic that I used for the authorization roles in Charlie’s security system. So the default-and-specific approach to templating was implemented easily, using the same database schema and the same sort of code.

Into the database I inserted the following test template:

<html>
    <head>
    </head>
    <body>
        <h1><asp:Label text="Welcome" runat="server" /></h1>
    </body>
</html>

The code to get this template onto the Page surface is incredibly simple. The code here executes within a code-behind or code-beside file (that is, within a class that inherits from System.Web.UI.Page).

String template = this.Template.Text;
Control parsed = this.ParseControl(template);
this.Controls.Add(parsed);

The above code will take the raw template string, parse it to end up with a handful of web server controls, and place those controls onto the Page surface. That part was easy, since the template above used built-in web server controls. The next part was to introduce custom server controls. Here is the next template I attempted to use:

<html>
    <head>
    </head>
    <body>
        <h1><custom:Label text="Welcome" runat="server" /></h1>
    </body>
</html>

The custom server control will give rise to the following exception:

Unknown server tag 'custom:Label'.

Normally, the .aspx page will have the following directive to “tell” it about custom controls:

<%@ Register 
    TagPrefix="custom" 
    Namespace="Charlie.Framework.Interface.Controls" 
    Assembly="Charlie.Framework" %>

However, even if that directive were to exist on the Page, such directives will not apply to any string of controls given to its ParseControl method. I had actually faced this same problem long ago when I was new to ASP.NET, and I posed this problem on the ASP.NET Forums. Fortunatey, Teemu Keiski gave me the answer. If the string that is passed into ParseControl contains custom tags, you must also pass in the directives. So, the string that is given to the ParseControl method must look like this:

<%@ Register 
    TagPrefix="custom" 
    Namespace="Charlie.Framework.Interface.Controls" 
    Assembly="Charlie.Framework" %>
<html>
    <head>
    </head>
    <body>
        <h1><custom:Label text="Welcome" runat="server" /></h1>
    </body>
</html>

The above will work, and the custom server control will be added to the Page. The remaining problem, however, is that Charlie cannot possibly know all of the directives that may be needed, since only the plugins will know. This is an easy problem to solve, since we have an existing IPlugin interface that defines those methods and properties that a plugin must implement in order for Charlie to “query” the plugin. The updated IPlugin interface looks like this:

public interface IPlugin
{
    Int32 ID { get; }
    String Name { get; }
    IWebModule[] GetModules();
    IWebResponseFilter[] GetResponseFilters();
    IWebContextFilter GetWebContextFilter();
    Presenter[] GetPresenters();
    IWebTemplateParser GetTemplateParser();
}

The last line is the new line. This means that Charlie can get the information it needs from the plugin in order to use custom server controls in the template strings that get passed to ParseControl. Here is what the IWebTemplateParser interface looks like:

public interface IWebTemplateParser
{
    String[] ControlDirectives { get; }
    String Parse(String template);
}

The first line of the interface allows the plugin to “give” to Charlie an array of strings, each of which will be a directive that needs to be added to a template before it is given to the ParseControl method. Here is a concrete example:

public String[] ControlDirectives
{
    get
    {
        String[] directives = new String[1];
        directives[0] = 
          @"<%@ Register 
                TagPrefix=""custom"" 
                Namespace=""Charlie.Framework.Interface.Controls""
                Assembly=""Charlie.Framework"" %>";
        return directives;
    }
}

And with this interface, the plugins can use any custom server controls that they want, and Charlie never needs to know about them. All that happens is that Charlie grabs the text of the template, and then says to each plugin, “If you use custom server controls, give me their directives.” Charlie then adds the directives to the front of the template, before passing it to the ParseControl method.

String template = this.Template.Text;
StringBuilder builder = new StringBuilder();
foreach (IWebTemplateParser parser in 
         PluginManager.Current.GetTemplateParsers())
{
    if (parser.ControlDirectives != null)
    {
        foreach (String directive in parser.ControlDirectives)
        {
            builder.Append(directive);
        }
    }
}
String directives = builder.ToString();
Control parsed = this.ParseControl(directives + template);
this.Controls.Add(parsed);

So that was the first problem to be solved in implementing the raw templating system: allowing for custom server controls. In the next weblog entry, I will explain the second part of the IWebTemplateParser interface.

by Alister Jones | Next up: Implementing Raw Templating - Part 2 →

Recursion is not a Dirty Word

2006-06-14T14:41:00.000-07:00

Here is an example of the “raw” templating system to be implemented by Charlie.

<html>
    <head>
    </head>
    <body>
        <insert:Snippet name="Header" />
        <h1><this:Title /></h1>
        <this:Content />
    </body>
</html>

If the Snippet to be inserted was simple text, then this templating system could be implemented with a series of find-and-replace operations. However, the inserted Snippet itself may include another <insert> element:

<div id="header">
    <h1>Company Name</h1>
    <insert:Snippet name="SearchBar" />
</div>

More troubling, that child Snippet may include an <insert> element that grabs not another Snippet, but a Text Resource:

<div id="searchbar">
    <insert:Text name="Search" />
    <input name="SearchTerm" />
</div>

This series of inserts within inserts within inserts is the age-old programming challenge of recursion. Fortunately, ASP.NET’s existing web server control system makes short work of recursion. ASP.NET allows us to place a single web server control onto a page, and that control may contain child controls. And those child controls may contain more controls. And so on.

Even better, ASP.NET provides a ParseControl method that allows us to pass in a string (exactly like the snippets shown above) and receive a Control in return. We can then plonk that control onto our web page, at which point ASP.NET’s page lifecycle will take over. In other words, this single method allows us to go from a simple string to a fully-functional, recursion-enabled web server control that can participate in all of ASP.NET’s goodies such as state management, event handling, and output rendering.

So while I have chosen to ignore ASP.NET’s built-in master pages, themes, skins, and so on, I will definitely make use of its web server control system. Anyone who has ever authored a custom web server control knows that this system is relatively simple, yet infinitely flexible. And you all know my goals for Charlie: keep it simple and keep it flexible.

by Alister Jones | Next up: Implementing Raw Templating - Part 1 →

The Raw Approach is Extensible

2006-06-14T02:25:00.000-07:00

A short while ago I introduced the story of three website owners: Tom, Michelle, and Joshua. Each owner wanted a different level of control over the look and feel of his or her website. Tom accepted a starting template provided by the Wizard. Michelle also accepted a starting template, but then used the Control Panel to customise the colours of the template. Joshua too accepted a starting template, but then updated the declarative templating system to customise all aspects of his website’s design.

It should be obvious that it is the Wizard that provides the default declarative templates. After the Wizard has been used to select the starting template, that starting template can be customised declaratively or through a graphical user interface. This was described in the previous weblog entry.

But how does the Control Panel fit into this templating system? Here is how the Colours may be defined through the Control Panel.

How then do we get these colour definitions into the declarative templating, which looks as follows?

<html>
   <head>
       <insert:Style name="Daylight" />
   </head>
   <body>
       <insert:Snippet name="Header" />
       <h1><this:Title /></h1>
       <this:Content />
       <insert:Text name="Disclaimer" />
   </body>
</html>

The first <insert> element is for a Style. The style will reside in a Styles folder, and its editor will look something like this:

What should be obvious is that the Style editor uses the same declarative approach to “inserting” external elements. To drive the point home, here is how our Snippet editor might look:

And where do these inserted Text elements come from? Well, just as the Control Panel provides the Colors icon, it would also provide a Text Resources icon. Here is how the Text Resources screen might look:

To summarize then, each <insert> element will “pull” information either from the Control Panel or from a special folder (such as Styles). Moreover, the inserted Snippet of HTML can do its own “pull” of information. So one Snippet can pull in another Snippet, which in turn pulls in a Text Resource, which in turn pulls in a Money Resource, and so on.

What this means is that for a website owner like Joshua who wants extensive control over how the website is designed, only a single declarative trick needs to be learned. That trick can then be applied over and over again. For a website owner like Michelle who wants limited control over the look of the website, only the Control Panel needs to be used—no need to learn the declarative trick. For a website owner like Tom who couldn’t care less about the look of the website, only the Wizard ever needs to be used to select a starting template.

At the end of the weblog entry titled A Dirty Sheet of Paper, I said that I took a pen and paper and said to myself, “Forget that you’re using ASP.NET. If you could create a user interface from scratch that allows a website owner to update both content and design, how would you do it?” The preceding weblog entries describe the templating system that I consider to be the most simple and the most flexible, with no consideration at all given to ASP.NET concepts such as master pages, themes, or skins.

After putting ASP.NET to one side and coming up with a theoretical templating system, it is time to bring ASP.NET back into play and come up with a working implementation of this templating system.

by Alister Jones | Next up: Recursion is not a Dirty Word

The Raw Approach is Simple and Flexible

2006-06-13T21:17:00.000-07:00

I finished my previous weblog entry by saying that I believed the simplest and most flexible way to style an article webpage would be to use a “raw” template, similar to the following:

<html>
    <head>
    </head>
    <body>
        <h1><this:Title /></h1>
        <this:Content />
    </body>
</html>

If you looked carefully at the last screenshot in that previous weblog entry, you would have noticed the following wording: “Note: Changes here will apply only to this single article. To update the template for all articles, edit the ‘ArticlePage’ template within the Templates folder.” If it is not already obvious, the idea is that a particular type of webpage (such as an Article, Weblog Entry, or Photo Gallery) would have a default template that would apply to all webpages of that type. The website owner is able to update these default templates.

In addition to being able to update the default templates, the website owner may override the template that will be applied to a specific webpage. If one article is a review of the latest album by Bruce Springsteen (I am not hip to any music released this century), then the website owner can update the template so that it includes a large image of Bruce before the article, and a graphical affiliate link following the article.

<html>
    <head>
    </head>
    <body>
        <p>
            <img src="/images/BruceSpringsteen.jpg" />
        </p>
        <h1><this:Title /></h1>
        <this:Content />
        <p>
            <a href="http://www.amazon.com?123456">
                <img src="/ads/Nebraska.gif" />
            </a>
        </p>
    </body>
</html>

The first benefit here is that a system of default-and-specific templates is very flexible. If you have ever tried to “tweak” a site-wide master page—so that a specific page is just a little bit different—you will know how frustratingly inflexible a master page system can be.

A second benefit here is that those website owners who want to dabble in HTML can do so. HTML is a wonderfully simple technology that is easy to learn, and it would be disrespectful to hide the HTML away as if to say to the website owner, “You’d just screw it up, so we’ve put it out of your reach.”

But what about website owners who do not care to dabble in HTML? There are two approaches that we can take to support website owners who want to control the look and feel of their websites, but who do not want to learn HTML.

The first approach is stay at the same “raw” level, but to abstract some of the HTML. Here is an example:

<html>
    <head>
        <insert:Style name="Daylight" />
    </head>
    <body>
        <insert:Image name="Bruce" />
        <h1><this:Title /></h1>
        <this:Content />
        <insert:Link to="www.amazon.com?12345"
                     image="/ads/Nebraska.gif" />
    </body>
</html>

What I like about this declarative approach is that we can extend it to support localized text.

<html>
    <head>
    </head>
    <body>
        <h1><this:Title /></h1>
        <this:Content />
        <insert:Text name="Disclaimer" />
    </body>
</html>

We can also extend this approach to insert common elements such as the headers and footers.

<html>
    <head>
    </head>
    <body>
        <insert:Snippet name="Header" />
        <h1><this:Title /></h1>
        <this:Content />
        <insert:Snippet name="Footer" />
    </body>
</html>

We can also extend this approach to insert elements from other plugins. Here is how a homepage might look.

<html>
    <head>
    </head>
    <body>
        <insert:Snippet name="Header" />

        <h1><insert:Text name="WebsiteTitle" /></h1>
        <h2><insert:Text name="WebsiteSummary" /></h2>

        <insert:Text name="RecentWeblogEntries" />
        <weblog:Listing recent="5" />

        <insert:Text name="RecentPhotos" />
        <photos:Gallery recent="5" />

        <insert:Snippet name="Footer" />
    </body>
</html>

Remember now, most of this template stuff was automatically created by the Wizard of a turnkey website, or by the developer of a custom website. The website owner does not need to touch this simple declarative templating unless he or she wants to make changes.

But what about the website owner who wants to control the look and feel of the website, but does not want to muck about in this declarative templating? The alternative approach is to introduce a pretty graphical user interface.

What is notable here is that this graphical user interface builds upon the declarative templating. Why is this notable? Well, if we had started with a GUI, we would have used logic to generate the resulting page:

Homepage homepage = GetHomepage();

TitleLabel.Text = homepage.Title;
SummaryLabel.Text = homepage.Summary;

if (homepage.ShowWeblog)
{
    WeblogEntryCollection entries =
        Weblog.GetRecentEntries(homepage.WeblogNumber);
    WeblogGridView.DataSource = entries;
    WeblogGridView.DataBind();
}

if (homepage.ShowPhotos)
{
    PhotoCollection photos =
        Photo.GetRecentPhotos(homepage.PhotoNumber);
    PhotoGridView.DataSource = photos;
    PhotoGridView.DataBind();
}

If we had started with the GUI and ended up with code like that above, we’d have one hell of a hard time retro-fitting a templating system that allows a website owner to make little changes here and there.

However, we did not start with the GUI. Rather, we finished with the GUI, where it simply provides a way of cloaking the declarative template. The template is still there, but it is hidden behind the GUI. But because the template is still there, and because the default-and-specific templating system is still there, we have not sacrificed the flexibility of the original “raw” approach to styling a webpage. If the website owner uses the GUI to define the default template for articles, he or she can still override the template that will be applied to a specific article. Now we have the friendliness of a GUI for the default template, and the flexibility to hand-craft the template for specific pages.

With this approach, we have now accounted for each of the three website owners in our little story. Tom was satisfied with the Wizard. Michelle was satisfied with the Control Panel. Joshua is satisfied with this flexible templating system.

What I have not yet discussed is how the settings in the Control Panel will apply to the declarative templating system. I’ll discuss that in the next weblog entry, as it highlights the flexibility provided by this declarative templating system.

by Alister Jones | Next up: The Raw Approach is Extensible →

A Raw Approach to Styling Webpages

2006-06-13T18:32:00.000-07:00

At the end of a recent weblog entry, I said that I had given myself the challenge, “Forget that you’re using ASP.NET. If you could create a user interface from scratch that allows a website owner to update both content and design, how would you do it?”

I started by drawing the user interface by which the website owner can compose an article. Mocked up in Photoshop, the user interface looks like this.

As an aside, the reason I have used Apple-like bubbles is that I am thinking ahead to the final user experience. The idea I am toying with is that when such a button is clicked, the icon animates to show that something is happening. This will be doubly-important if I introduce AJAX elements. So I’m using bubbles simply because that is where my thoughts are presently. This style may or may not find its way into Charlie’s final user interface.

Coming back to the topic of this weblog entry, you will see the Show Template button at the bottom of the article editor. Clicking on it will reveal the following additional interface elements.

This was the idea I had when I put aside my preconceptions of master pages, templates, skins, themes, and everything else. As you will see, it is a very “raw” approach to styling a website. My contention is that anything less raw will be less flexible. It is also my contention that anything less raw will be less simple. In other words, I believe this raw styling mechanism represents the most simple and the most flexible option available. I’ve given this idea a lot of thought, so I’ll dedicate a few weblog entries to this “raw” approach to styling.

by Alister Jones | Next up: The Raw Approach is Simple and Flexible →

Olive, Teal, Peach, and Mauve

2006-06-12T18:24:00.000-07:00

In our story of Tom, Michelle, and Joshua, we have seen how each has used a Wizard to choose the starting template for his or her website.

Tom has no interest in customising the appearance of his website, so he’s happy to run with one of the templates. That is, the template selection presented by the Wizard is all the customization that Tom wants.

Michelle wants to present her photography against a black background, with all text in mauve. We can presume that she will use the Wizard to choose a template that is close to the look she wants. Now, how can we allow Michelle to “tweak” the template to her liking? She doesn’t want to change the layout of the template—just the colours used.

Let’s consider how we might allow Michelle to change the colours of her website. In order to provide Michelle with a happy user experience, we should enable Michelle to change the colours of the website in the way that she expects to be able to change the colours of the website.

If I had elected to use the PowerPoint interface convention, then the most likely way to change the colours of the website would be through the Format menu. However, I earlier decided against using the PowerPoint interface convention.

My long-ago decision was to use an Explorer interface convention. I am still very comfortable with this decision, since an Explorer interface convention is familiar to users and flexible for Charlie. So, within this convention, where would a user look to change the colours of the website? Fortunately, both the Windows and the Macintosh operating systems place colour options in the same location: within the Control Panel.

At this point, I am relatively unconcerned with the design of the Control Panel. Still, the above looks fairly need and tidy, so Charlie may ultimately use a similar design. (The icons are based on those provided by Amit.)

Once Michelle has selected the Colours option, what should she see?

My first idea was that it would be very user friendly to have a dirt-simple preview pane that allows the website owner to see which colours he or she is changing. I started by mocking up the following illustration.

As I started working with this illustration, I thought to myself, “You’re creating a headache for yourself. How can you have a workable preview when templates can change, fonts can change, and anything and everything can change?” I also worried about how the colour palette (not shown, but would be there on the right) could “point” to the appropriate spot in the preview image. After thinking it through, I have decided that this is simply not a feasible approach.

I then performed a Google Image search on user interfaces that allow a user to select a colour palette. I could not find an interface that would support the flexibility I desired. I then fired up CityDesk. Here is its colour palette.

This is an extremely simple way of allowing users to select colours. Is it too simple? I don’t think so, and in fact I think its simplicity ties in neatly to the “styling” idea that I have. I will come back to this idea in the next few weblog entries. For now, I’d like to make just one comment on the above dialog.

You will notice that the dialog allows a palette of eight colours. CityDesk does not allow you to change this number. You cannot even change the names of the colours. This sort of rigidity would cause many developers to wear Doc Martins so that they can stomp around in protest. While I cannot speak for Joel Spolsky, I think he might point us to the common programmer thought pattern: there are only three numbers: 0, 1, and n. If n is allowed, all n’s are equally likely. In this case, if CityDesk or Charlie allows its users to specify two colours, the software should allow the users to specify 73 colours. I’ll let you read the Joel’s argument against that idea, since he can express himself much better than I can. I’ll just take the easy tack of saying, “I agree with Joel.”

What is interesting—if I may play fast and loose with the meaning of the word—is that this idea circles back to the concept of making opinionated software. Give the user a palette of eight colours, and be done with it. If the user wants to use more colours than that, we should send him or her a book about taste and style.

So Charlie is going to take its cue from CityDesk with respect to allowing the user to change colours. I will come back to CityDesk in the next few weblog entries, and you’ll see why I am not “thinking this one through” too much.

by Alister Jones | Next up: A Raw Approach to Styling Webpages →

Ready, Set, Go!

2006-06-07T21:57:00.000-07:00

In my previous weblog entry I introduced three characters to the story of Charlie. Tom, Michelle, and Joshua have each decided that they need a website, and have visited a Charlie-based website that will allow them to create a turnkey website. They have each clicked on the “create a new website” button. What should happen next?

What should happen next is whatever the visitor expects to happen next. And what will the visitor expect? It is my firm belief that the visitor will expect some sort of Wizard process to start. Every computer user has installed either an operating system, an application, a game, a utility, a driver, an internet connection, or something else. And nearly every such installation involves a Wizard process. In fact, anything except a Wizard process will be so unexpected as to feel somehow “wrong”. So I am not even going to think about alternative approaches. Moreover, I think that the Wizard approach is one of the better software interface innovations. A Wizard it will be.

The first thing that the Wizard should do is have Charlie say hello. Long ago I made the argument that Charlie should be visible to the user. It seems to me that the first step of the Wizard process should be for Charlie to introduce itself.

I did consider making the first Wizard step devoid of having the user enter any information. But that would just drag out the sign-up process with a useless, if polite, first step. Worse, a long introduction might smack of salesmanship, and nobody likes a salesman.

Right now, I am not too concerned with how the Wizard works. The point of this exercise is to find the simplest way for Tom, Michelle, and Joshua to get going with the look and feel of their turnkey websites. So let’s just presume that the next screen asks the user to provide a password.

Let us also presume that the following step is for the user to choose his or her username.

Now we come to the heart of the matter. Tom just wants to get going with his turnkey website. Michelle wants to some control over how her website looks. Josha wants great control over how his website appears. Should we ask the user how much control he or she wants? Not only would that be a silly question, it would suggest to the user that once he or she has made a choice, that choice cannot be undone. A much better approach is to give the user some starting options, and let the user know that the choice can be changed.

Providing starting templates is a common approach in many software applications. Typically, those starting templates provide a base from which the user can either change the template or use it as a starting point for extensive customisation. In other words, this is a common approach that precisely reflects the customisation available to the Charlie user.

A starting template is helpful to Tom, Michelle, and Joshua, even though they each have very different customisation needs. So this will be the approach taken with Charlie.

In retrospect, this idea of a starting template is so common that it seems comical that I have given it any consideration. But stay with me, because we still need to determine how this templating system will be implemented in Charlie. Will it use the MasterPage system of ASP.NET version 2.0, use a custom system based on User Controls, or use some other system altogether?

by Alister Jones | Next up: Olive, Teal, Peach, and Mauve →

A Clean Sheet of Paper

2006-06-07T19:44:00.000-07:00

In part two of his four-part article series on Painless Functional Specifications, Joel Spolsky recommends that we engage in a little bit of story telling. We should come up with a few fictional, but realistic, users of our software product, and tell the story of how the users work with our product. To put a slight spin on this suggestion, we might say that the story should describe how the users want to work with our product. Then, we should design the product to get as close as possible to having it work the way our users want it to work.

I am going to tell the story of how three website owners will want to work with Charlie.

The first website owner is named Tom—a no-nonsense name for a no-nonsense guy. All Tom wants is to click a button or two and have a website that is ready to go. He doesn’t want to be worried about templates and colours and other shit. He just wants a website where he can start writing articles about his favourite subject, Carrol Shelby.

The second website owner is named Michelle—an amateur photographer who wants to show her photos to the world. She just wants a website that presents a few albums where each photo can be clicked in order to see the full-sized photo. Standard stuff. But whereas Tom doesn’t care about templates and colours, Michelle does. She wants the background to be black, since most of her photography is black and white. She’d like the title and the photo borders to be mauve, which is her favourite colour.

The third website owner is named Joshua—the owner of a boutique winery. He has paid a lot of money to a graphic design firm to come up with a design for his wine labels, so he wants his website to reflect the same style. His labels display titles in a Garamond typeface, flush right, with the first letter in red and the remaining letters in black. The website should use the same style.

One of the originally-specified features for Charlie was that it would support both custom websites and turnkey websites.

A custom website would come about by Tom, Michelle, or Joshua sending me an email or calling me on the phone and saying, “I hear you’re the second-greatest website designer in the world. Jason Santa Maria is unavailable, so I’d like for you to design my website.” Creating a custom website is relatively easy—it just takes time. It is the turnkey websites that require special attention.

A turnkey website is one where the website-creation process is automated. This would come about by Tom, Michelle, or Joshua visiting a Charlie-based website and clicking on the “create a new website” link. Creating a turnkey website is relatively hard—it requires designing a system with the right tradeoffs between automation and customization. Tom wants it all automated, Michelle wants a little customization, and Joshua needs extensive customization.

In the following weblog entries on creating the user experience, I’ll be looking only at the turnkey experience, since that is the one that requires planning and coding. Any custom website will really just be a single-instance turnkey site with lots of customization. In other words, planning and coding for turnkey websites will lay the foundation for custom websites too.

So, having forgotten that Charlie just happens to use ASP.NET—and thereby starting with a clean sheet of paper—how can the product be designed to provide Tom, Michelle, and Joshua with a pleasant and effective experience in creating their turnkey websites?

by Alister Jones | Next up: Ready, Set, Go! →

A Dirty Sheet of Paper

2006-06-06T20:20:00.000-07:00

Over the last couple of days, I have introduced to Charlie a few of the goodies from ASP.NET version 2.0. Namely, the application can now use MasterPages and the Login server control. I already had a simple templating system and a simple login control, but I wanted to keep Charlie consistent with the direction that ASP.NET is taking. So I replaced my simple versions with the built-in versions. Then, after finding the built-in versions of templating and the login control harder to work with, I reverted back to my simple versions.

A few days earlier I had needed to update the HTML of one small part of just one page. The existing, simple templating system made this much harder than it should have been. And the new ASP.NET MasterPage system would have made it even harder again.

A few weeks ago I attempted to skin Community Server. My recent motorbike accident was less painful. Many months ago I had abandoned my weblog at AspAdvice.com because I could not take control of how Community Server styled my weblog. By contrast, the templating system used here at Blogger is much better.

These experiences made me realise that I am approaching Charlie’s interface layer with a number of preconceptions—the sheet of paper is not clean, but dirty.

In addition to my preconceptions about “how skinning is done in ASP.NET,” I am also bringing forward preconceptions about how data should be entered on a webpage.

One year ago I worked as an editor of a website, and the website used a clunky old rich text box. I spent many, many hours undoing all of the HTML junk the rich text box introduced. It was a terrible waste of time.

A little before that experience, I worked on the design of a website, which was then handed over to a web development company to implement in their content management system. It turned out that the CMS, which they trumpeted as being state-of-the-art, was nothing but a free and simple rich text box. They just slapped the HTML into a database, and gave the client a rich text box with which to edit the content. The problem was, by not separating the content from the design, the client kept inadvertently destroying the design of the site.

As a result of this experience, I have an additional preconception that “rich text boxes are bad.”

Yesterday I decided that I needed to start with a clean sheet of paper. I sat on a couch with a pen and paper and said to myself, “Forget that you’re using ASP.NET. If you could create a user interface from scratch that allows a website owner to update both content and design, how would you do it?”

by Alister Jones | Next up: A Clean Sheet of Paper →

“You Are Here”

2006-06-05T20:23:00.000-07:00

Most ASP.NET applications have many files with the extension, .aspx. If the visitor requests the page about.aspx, then ASP.NET finds the file with that name, loads it up, executes it, and returns the HTML to the visitor. If the user requests the page contact.aspx, ASP.NET will find the file with that name, load it up, execute it, and return the HTML.

Some ASP.NET applications take a different approach. Rather than having a whole bunch of .aspx files, they have just one; typically it will be named default.aspx. What these applications do is use this single default.aspx file to “build” a dynamic page. So even if the visitor requests about.aspx, the application secretly redirects the request to default.aspx, and then builds the About page. If the visitor requests contact.aspx, the application again secretly redirects the request to default.aspx, and then builds the Contact page.

Charlie takes an altogether different approach. There is no .aspx file anywhere in the Charlie application. Instead, a page request will be handled by the WebHandler object, which exists only in memory. There is no corresponding .aspx file.

While I was in the early stages of developing Charlie, this approach worked fine. But as development progressed, this approach introduced two problems.

The first problem is that I could not get the Page.ParseControl method to work. Charlie kept complaining about its VirtualPathProvider. I told myself not to worry about this problem, because I was not yet at the stage of working with Charlie’s Interface layer. So I fudged my way around it.

Today I tried to introduce MasterPages to Charlie. The code is simple:

Page.MasterPageFile = "~/Templates/Default.master";

But Charlie was having none of it. Once again it started complaining about its VirtualPathProvider. This time, I could not fudge my way around the problem—I had to work it out.

If I created a new web application and put the above code into the code-file of an .aspx page, it worked fine. But if the “page” exists only in memory, as with Charlie, it does not work. I could not figure out why. The above code clearly states where the master page file is located, yet Charlie would not load it up. Why?

It turns out that a new feature of ASP.NET version 2.0 was throwing a spanner in the works. With ASP.NET version 1.1, a tilde-based path (such as “~/Templates”) would always resolve from the application’s root folder. This is precisely what the tilde means, so this was precisely the expected behaviour. However, with ASP.NET version 2.0, a tilde-based path may or may not resolve from the application’s root folder. ASP.NET version 2.0 allows an application to change how a tilde-based path is resolved. David Ebbo provides a weblog entry on how to do this. Ironically, I am the customer to whom David refers.

The problem for Charlie was that because its WebHandler object has no corresponding .aspx file, it did not know how to resolve a tilde-based path. In other words, Charlie did know know where this “virtual page” was located within the file system. The solution to the problem then was to tell the Charlie page, “You are here.”

Page.AppRelativeVirtualPath = @"~/";

This tells Charlie to consider that its virtual page is located within the application’s root folder. With that single line of code, Charlie has stopped complaining about its VirtualPathProvider. And I have stopped swearing at Charlie—at least for the time being.

by Alister Jones | Next up: A Dirty Sheet of Paper →

The Valley of Data Access - Part 6

2006-06-05T05:25:00.000-07:00

Done. The database access code for Charlie has now been refactored.

All duplicate code has been pulled out of the many Mapper classes and moved into a single Helper class. The Helper class has also taken on the responsibility of performing the cascading security and localization work. Even better, the cascading logic no longer requires repeated trips to the database. Where previously Charlie required an embarrassing 49 hits to the database in order to serve the first webpage, it now requires twelve. Subsequent page requests require about eight hits.

Charlie currently hits the database once to retrieve an business entity or an entity collection, and then hits the database again to retrieve the security roles for the entity or collection. As I mentioned in my last weblog entry, I had a go at combining these two queries into one. I believe it can be done, but the approach involves a few penalties. The first penalty is that I’d have to swap from using fast DataReaders to relatively slow DataSets. The second, and greater, penalty is that I’d have to introduce a tight coupling between Charlie and its Security plugin, so that they can “gang up” their database queries. The third penality is that ganging up the queries would make it awkward for a plugin to use a different data store. Maybe the Weblog plugin could make use of the free MySQL database available on my WebHost4Life account, while the Security plugin uses the Microsoft SQL Server database. That flexibility appeals to me.

So I have put aside the idea of trying to combine the queries. If the extra database queries ever become a problem, that is the time I will look again at the issue. And even if the issue returns, a hardware solution might prove to be better than a software solution. If a website ever becomes so highly trafficked that the extra hits incur a major performance penalty, then that website might warrant a dedicated server. But again, it is a problem only when it becomes a problem.

One slight improvement to Charlie’s data access code is the introduction of a SqlConnectionManager. Previously, each Mapper created, opened, used, and then closed a new connection. Now, each Mapper pulls an open database connection from the SqlConnectionManager and then, when it’s finished with the connection, returns it to the Manager. So that Charlie doesn’t end up hanging on to open connections for too long, the SqlConnectionManager receives a call to its CloseConnections method at two points in the ASP.NET page lifecycle. The first point is before the HttpHandler starts to execute, and the second point is when the page request has finished processing. What this means is that the absolute longest that Charlie holds an open database connection is 0.4 seconds, and it is usually much more brief. Here is the code for the simple SqlConnectionManager:

internal static class SqlConnectionManager
{

   private static String contextKey = 
      "Charlie.Framework.Services.DataAccess.SqlConnectionManager";

   internal static SqlConnection GetConnection(String connectionString)
   {
      HttpContext context = HttpContext.Current;
      Hashtable connections = context.Items[contextKey] as Hashtable;
      if (connections == null)
      {
         connections = new Hashtable();
         context.Items[contextKey] = connections;
      }
      SqlConnection connection = 
         connections[connectionString] as SqlConnection;
      if (connection == null)
      {
         connection = new SqlConnection(connectionString);
         connections.Add(connectionString, connection);
      }
      if (connection.State != ConnectionState.Open)
      {
         connection.Open();
      }
      return connection;
   }

   internal static void ReturnConnection(SqlConnection connection)
   {
      // nothing yet
   }

   internal static void CloseConnections()
   {
      HttpContext context = HttpContext.Current;
      Hashtable connections = context.Items[contextKey] as Hashtable;
      if (connections != null)
      {
         IEnumerator enumerator = connections.GetEnumerator();
         while (enumerator.MoveNext())
         {
            DictionaryEntry entry = (DictionaryEntry)enumerator.Current;
            SqlConnection connection = entry.Value as SqlConnection;
            if (connection != null)
            {
               if (connection.State != ConnectionState.Closed)
                  connection.Close();
               connection = null;
            }
         }
         connections.Clear();
      }
   }
}

With the exception of the first page request and its inexplicable two-second delay, Charlie is taking between 0.02 and 0.4 seconds to serve a page request. While the slower responses are always due to database activity, the response times still seem fast enough. I am not going to try to optimise the code further.

So the bulk of Charlie’s Business layer and Persistence layer is done. I’m now going to move onto the Controller layer. That should be easy, because I am going to pirate some code. Stay tuned for some swashbuckling tales.

by Alister Jones | Next up: “You Are Here” →

The Valley of Data Access - Part 5

2006-06-04T10:26:00.000-07:00

A two-letter word has had a profound impact on Charlie’s database access code. Before I learned of this word, I did not know how to tell SQL Server to test for one of a range of values. So, I took the brute-force approach of simply issuing the same query over and over, passing in new parameter values each time.

Here is the text of the command that I needed to execute:

String query =
      @"SELECT
            r.Role_Id, 
            Role_Name, 
            DomainEntityRole_CanCreate, 
            DomainEntityRole_CanRetrieve, 
            DomainEntityRole_CanUpdate, 
            DomainEntityRole_CanDelete
        FROM
            Charlie_DomainEntityRole der
            INNER JOIN
                Charlie_Role r
            ON
                r.Role_Id = der.Role_Id
        WHERE
            EntityType_Id = @entitytypeID
        AND
            Entity_Id = @entityID
        AND
            Domain_Id = @domainID";

Then, in order to test different parameter values, I issued three separate requests for a DataReader:

command.Parameters.Clear();
command.Parameters.AddWithValue("@entitytypeID", entityTypeId);
command.Parameters.AddWithValue("@domainID", domainId);
command.Parameters.AddWithValue("@entityID", entityId);
reader = command.ExecuteReader();
triplet.CollectionByEntityId = FillRoleCrudCollection(reader);
reader.Close();

command.Parameters.Clear();
command.Parameters.AddWithValue("@entitytypeID", entityTypeId);
command.Parameters.AddWithValue("@domainID", domainId);
command.Parameters.AddWithValue("@entityID", -1);
reader = command.ExecuteReader();
triplet.CollectionByDomainId = FillRoleCrudCollection(reader);
reader.Close();

command.Parameters.Clear();
command.Parameters.AddWithValue("@entitytypeID", entityTypeId);
command.Parameters.AddWithValue("@domainID", -1);
command.Parameters.AddWithValue("@entityID", -1);
reader = command.ExecuteReader();
triplet.CollectionByEntityTypeId = FillRoleCrudCollection(reader);
reader.Close();

If this were a rare requirement, then this brute-force approach might be acceptable. However, this was the code that implemented the cascading logic needed for each entity to receive its security roles. So each time an entity was requested from the database, this silly code issued a further three queries.

Fortunately my Google searching turned up a little gem of a tutorial: Introduction to Structured Query Language by James Hoffman. Included in the tutorial is a brief example of the IN keyword, and with that example I was able to undo the silliness above. Here is the updated command text:

String query =
      @"SELECT
            r.Role_Id, 
            Role_Name, 
            DomainEntityRole_CanCreate, 
            DomainEntityRole_CanRetrieve, 
            DomainEntityRole_CanUpdate, 
            DomainEntityRole_CanDelete,
            Domain_Id,
            Entity_Id
        FROM
            Charlie_DomainEntityRole der
            INNER JOIN
                Charlie_Role r
            ON
                r.Role_Id = der.Role_Id
        WHERE
            EntityType_Id = @entitytypeid
        AND
            Entity_Id IN (@entityID, -1)
        AND
            Domain_Id IN (@domainID, -1)";

With that change it now takes only one database hit to retrieve the roles for an entity. Fortunately, it did not take too long for me to realise that this exact same query could be used to retrieve the roles not just for a single entity, but also for a collection of entities. Rather than passing in a single @entityID parameter, I would pass in the ID values of all the entities in the collection:

String IDmarker = "[[@entityIDs]]";
String query =
      @"SELECT
            r.Role_Id, 
            Role_Name, 
            DomainEntityRole_CanCreate, 
            DomainEntityRole_CanRetrieve, 
            DomainEntityRole_CanUpdate, 
            DomainEntityRole_CanDelete,
            Domain_Id,
            Entity_Id
        FROM
            Charlie_DomainEntityRole der
            INNER JOIN
                Charlie_Role r
            ON
                r.Role_Id = der.Role_Id
        WHERE
            EntityType_Id = @entitytypeid
        AND
            Entity_Id IN (" + IDmarker + @")
        AND
            Domain_Id IN (@domainid, -1)";
StringBuilder builder = new StringBuilder();
for (Int32 i = 0; i < criteria.EntityIDs.Count; i++)
{
    String param = String.Format("@entityid{0}", i.ToString());
    builder.AppendFormat("{0},", param);
    command.Parameters.AddWithValue(param, (Int32)criteria.EntityIDs[i]);
}
builder.Append("-1");
query = query.Replace(IDmarker, builder.ToString());
command.CommandText = query;
command.Parameters.AddWithValue("@entitytypeid", criteria.EntityTypeId);
command.Parameters.AddWithValue("@domainid", criteria.DomainId);

Previously, a collection of ten entities would require 30 database hits in order to retrieve all the roles for that collection. Now, no matter how many entities are within the collection, only a single database visit is required to retrieve the security roles.

That’s a great improvement, but I am still a little disturbed that one database visit is required to retrieve an entity or an entity collection, and then a separate database visit is required to retrieve its roles. To state the bleeding obvious, this approach is doubing the number of hits to the database.

I have been in two minds about whether this is a problem that needs to be solved. While an extra database hit is of course undesirable, performance is not everything. The current design cleanly separates entity content from entity security.

What I have resolved to do is to have a bash at combining the two queries into one. In most circumstances I would say to myself, “Until it actually becomes a problem, it is not a problem to be solved.” However, I need to get a better understanding of SQL, so I consider this to be an exercise that may also provide a performance boost for Charlie. I’ll give it a go, but I won’t be too concerned if I cannot get it to work.

by Alister Jones | Next up: The Valley of Data Access - Part 6 →

Charlie Wears Lead Boots

2006-06-03T02:51:00.000-07:00

I finished my previous weblog entry by saying that I would teach myself enough SQL so that I could move a little bit of logic to the database, rather than keeping the logic in the C# code and hitting the database multiple times.

Fortunately, a friend has offered to help me with writing the queries, and I’ll report on how that goes. So that we focus on the most “chatty” parts of Charlie’s database access, I decided that I should use SQL Profiler to see what is going on at the database. I “touched” the Global.asax file in order to mark the Charlie application for recycling, and requested the home page. I switched to SQL Profiler and looked at its trace. I was horrified.

To start with, the trace showed signficantly more hits to the database than should have occurred. Since this is an honest account of Charlie’s development, I’ll give you the true figure, as much as I’d like to halve it in order to save some face. The first page request required 49 hits to the database. Just as troubling, there was a two-second delay between two subsequent database hits. What the heck was Charlie doing in that time?

I switched on Charlie’s logging plugin, cleared the SQL Profiler trace, and again forced the Charlie application to recycle. After requesting the home page again, I now had a record of what Charlie was doing, and what the database was doing. I brought the details together in an Excel spreadsheet, and analysed what was going on.

The first problem was that Charlie was issuing the same queries over and over. In fact, I counted 23 redundant queries. This is a coding error on my part, and some good old-fashioned debugging will solve it. So, while I am suitably embarrassed, I am not too worried about this.

The second problem was that Charlie was issuing three queries that should really be combined into one query. This is the logic problem that I prompted this investigation. Hopefully, my friend can help me here.

The third problem was the Charlie was querying once for each entity, and then once again for each entity’s roles. Performing two queries for each entity seems unnecessary, so I may look at combining the queries.

The fourth problem was the database connection was opened and closed eight times. It is my understanding that there is some overhead involved in opening a new connection, so I may consider keeping a connection open when I know another query is imminent.

If I address each of the above four problems, I should be able to get the initial database access down to about six queries and two connections. That will solve the “chatty” database access problem. But, there was a bigger problem.

The fifth problem was that the SQL Profiler trace showed a two-second gap between two adjacent queries. The output of the logging plugin shows where the delay occurs, even though I don’t know the cause of the delay.

Charlie’s extension of the HttpApplication class (the class behind the Global.asax file) logs the time immediately before the target HttpHandler receives its ProcessRequest command. Then, that HttpHandler logs the time when it starts processing the request. The log shows that there is a 2.1 second delay. Charlie is not doing anything at all during this time. ASP.NET is doing something. What the hell is it doing that is taking 2.1 seconds?

First, it is not a logging error, because the same delay is recorded by both Charlie and SQL Profiler. The delay is real.

Second, it is not a delay caused by ASP.NET parsing an .aspx file and storing the dynamically-generated DLL. The HttpHandler used by Charlie is a ready-to-go class that is fully contained within an assembly.

Third, this delay only occurs during the first page request after recycling the ASP.NET application. However, it is not the time taken to restart the application, since both the logging plugin and SQL Profiler do not come into play until after the application has started and the request has commenced processing.

What is ASP.NET doing that takes 2.1 seconds? The HttpHandler is loaded and ready to go, but there is a marked delay before anything happens.

To put this delay in context, the request for the Home page after an application restart takes 3.1 seconds. A subsequent request for the About page (so there are a few database hits for the new content) takes 0.1 second. A new request for the Home page (so everything is drawn from cache) takes 0.03 seconds. This inexplicable 2.1 second delay during the processing of the first request is far and away the slowest part of Charlie.

As I write this, I have no idea why there is a long delay before the first HttpHandler executes. I fully understand the delay caused by the ASP.NET application restarting and reloading. But I do not understand what ASP.NET is doing from the time it fires the PreRequestHandlerExecute event to the time the handler actually executes. If I learn the cause of the delay, I will of course let you know. And if you know the cause, you will of course let me know, won’t you?

By the way, I know that optimisation should be a final polishing step. But, this investigation did not come about due to a desire to optimise Charlie, but came about as a consequence of using SQL Profiler to address Charlie’s chatty database access code.

by Alister Jones | Next up: The Valley of Data Access - Part 5 →

The Valley of Data Access - Part 4

2006-06-02T08:21:00.000-07:00

In the previous weblog entry I set out the plan of how I would refactor the database access code. I am pleased to say that, with just one variation, the refactoring went as planned.

The variation was reasonably cosmetic. In the original plan, the Mapper would gather together the raw query text and a collection of parameters, and pass them off to the new Helper class. This plan was a little short-sighted, as future developments may mean that the Mapper wants to pass the name of a stored procedure to its Helper class, rather than pass the raw query text. So, to allow for future flexibility, I created a new EntityCommand class that looks like this:

namespace Charlie.Framework.Persistence
{
    public class EntityCommand
    {
        public String CommandText
        {
            get
            {
                return this.commandText;
            }
            set
            {
                this.commandText = value;
            }
        }
        private String commandText;

        public EntityParameterCollection Parameters
        {
            get
            {
                return this.parameters;
            }
        }
        private EntityParameterCollection parameters = 
            new EntityParameterCollection();
    }
}

So, rather than passing the raw query text and parameters to its Helper, the Mapper passes a high-level EntityCommand object. In the future, I can extend this EntityCommand object to expose a CommandType property, allowing the Mapper to tell its Helper that a stored procedure is to be used.

With the refactoring complete, the long Retrieve method shown in the second entry in this series has been reduced to this:

public override Entity Retrieve(Entity entity, EntityCriteria crit)
{
    NoteCriteria criteria = (NoteCriteria)crit;

    EntityCommand command = new EntityCommand();
    command.CommandText =
          @"SELECT
                s.Note_Id, 
                Note_CreatedBy, 
                Note_CreationDate, 
                Note_UpdateDate, 
                NoteLocalized_Title, 
                NoteLocalized_Content
            FROM
                Charlie_Note s
            INNER JOIN
                    Charlie_NoteLocalized sc
                ON
                    s.Note_Id = sc.Note_Id
            WHERE
                s.Note_Id = @noteid
            AND
                sc.NoteLocalized_Culture = @culture";
    command.Parameters.Add("@noteid", criteria.Id);
    command.Parameters.Add("@culture", criteria.Culture);

    Note note = (Note)this.Helper.Retrieve(
                entity, criteria, command, this.CreateNoteFromReader);

    return note;
}

All of the red code has been moved out to the Helper class. Better yet, all that red code is shared amongst all of the Mapper classes, where before it was duplicated across those classes.

The second-last line of code includes a delegate that points to the following method of the Mapper:

private Entity CreateNoteFromReader(IDataReader reader)
{
    Note note = new Note();
    note.Title = Convert.ToString(reader["NoteLocalized_Title"]);
    note.Content = Convert.ToString(reader["NoteLocalized_Content"]);
    note.CreationDate = Convert.ToDateTime(reader["Note_CreationDate"]);
    note.UpdateDate = Convert.ToDateTime(reader["Note_UpdateDate"]);
    return note;
}

What you may notice is that the only code that is left in the Mapper is the code that actually does the mapping of the database fields to the business object properties. Everything else has been relocated to the Helper class. If I was an OOP nerd, I might suggest that the cohesion of the class has been increased. But as I am an OOP newbie, I’ll just say that the class is now simpler.

The exercise of refactoring the database access code did bring to light a weakness with Charlie, which is actually a weakness with me. The code in the RoleMapper class could not be refactored to the new system, because its Retrieve method hits the database three times, whereas all other Retrieve methods hit the database one time. The only reason the code hits the database three times is because I could not figure out a more effective SQL command.

So guess what I’m going to learn next?

by Alister Jones | Next up: Charlie Wears Lead Boots →

The Valley of Data Access - Part 3

2006-06-01T09:17:00.000-07:00

At this point in the life of Charlie, I am refactoring the data access code. In the previous weblog entry, I explained why my approach is to extract from the data access methods the parts that stay the same, and leave the mappers to concentrate on their unique requirements. Staying with the earlier example of a Retrieve method, I see four unique parts that the Mapper must gather together before passing them off to the Helper to execute.

Starting from the top of the method and working down, the first unique part is the entity that is being retrieved.

Document document = (Document)entity;

The second unique part of the Retrieve method is the query string:

String query =
   @"SELECT
         d.Document_Id,
         d.Document_ParentId,
         d.Document_Name,
         d.Document_FriendlyUrl,
         d.Document_Position,
         d.Document_CreationDate, 
         d.Document_UpdateDate, 
         c.DocumentLocalized_Culture,
         c.DocumentLocalized_Title
     FROM
            Charlie_Document d
         JOIN
            Charlie_DocumentLocalized c
         ON
            c.Document_Id = d.Document_Id
     WHERE ";
if (criteria.LoadById == true)
{
   query += " d.Document_Id = @documentid";
}
else if (criteria.LoadByUrl == true)
{
   query += " d.Document_FriendlyUrl = @friendlyurl";
}

Now, a seasoned developer would be horrified at the appearance of a hard-coded query like that. Personally though, I consider its clumsiness to be offset by three compelling benefits. First, it communicates clearly the query that will be executed. Second, it allows me to copy the query into Query Analyser, test it and perhaps tweak it, and paste it back into Charlie’s code. Third, it’s simple. The seasoned developer can add attributes to the business objects or add mapping rules to an XML file. Charlie and I will just slap the query in place.

The third unique thing that the Retrieve method must gather together is the collection of command parameters. Currently, the code looks like this:

if (criteria.LoadById == true)
{
   command.Parameters.AddWithValue("@documentid", criteria.Id);
}
else if (criteria.LoadByUrl == true)
{
   command.Parameters.AddWithValue("@friendlyurl", criteria.Url);
}

This just needs to be updated slightly so that the mapper gathers together a single collection of parameters:

ParameterCollection parameters = new ParameterCollection();

if (criteria.LoadById == true)
{
   parameters.Add(new Parameter("@documentid", criteria.Id));
}
else if (criteria.LoadByUrl == true)
{
   parameters.Add(new Parameter("@friendlyurl", criteria.Url));
}

I will use a custom Parameter class, because I want to be able to define “alternative” values in order to support the cascading logic used throughout Charlie. Using localization as an example, the following may be a custom parameter:

Parameter cultureParameter = new Parameter("@culture", "fr-FR", "fr", "en");

The final unique thing within each mapper’s Retrieve method is seen in the bold line below:

if (reader.Read())
{
   document = NewDocumentFromReader(reader);
}

The document was the first unique thing gathered by the Mapper, so we have that part. But what can we do about the second part? How can we tell the Helper method that once it has executed the passed-in query and obtained the resulting data reader, we want that reader to be passed to our NewDocumentFromReader method?

What would seem to be an obvious solution would be to have the Helper simply return the reader to the calling Mapper code. The Mapper code then passes the reader to the NewDocumentFromReader method. However, the Mapper is then left holding a darn data reader, which it must tidy up. But the whole point of this refactoring exercise is to free the Mapper from having to worry about data connections, transactions, readers, exceptions, and everything else.

Fortunately there is another solution. Just as the Mapper can pass the entity object, the query string, and the parameter collection to the Helper class, it can also pass an entire method to the Helper class. In truth, it does not pass the method itself but rather a delegate of the method. Teemu Keiski describes this process in his article, Using Delegates with Data Readers to Control DAL Responsibility. If the concept of delegates is new for you, I wrote a tiny tutorial on delegates on the ASP.NET Forums.

We have seen that the Retrieve method in each EntityMapper needs to gather together four objects to send to its Helper class to execute. The skeleton of the Retrieve class therefore looks like this:

public override Entity Retrieve(Entity entity, EntityCriteria crit)
{
    // Get the entity
    Document document = (Document)entity;
    
    // Get the query
    String query = @"SELECT ... ";
    
    // Get the parameters
    ParameterCollection parameters = new ParameterCollection();
    parameters.Add(new Parameter("@name", value));
    
    // Get the delegate
    IReaderHandler handler = new IReaderHandler(NewDocumentFromReader)); 
    
    // Pass them off to the Helper for executing
    this.Helper.Retrieve(document, query, parameters, handler);

    return document;
}

This trim Mapper.Retrieve method means that all the red code in the previous weblog entry has been extracted to the Helper.Retrieve method. This Helper.Retrieve method assumes responsibility for the database connection, any transactions that may be in progress, any readers that are generated, any exceptions that are thown, and any logging that may be required. Since all the Mappers will use this Helper.Retrieve method, there is now a single point at which the data access code can be tweaked or corrected.

Now, as I said in the first entry on refactoring the data access code, I am writing this series of weblog entries “live”—I have not yet performed this refactoring.

The next step is to actually do the refactoring. I will of course report on any surprises along the way.

by Alister Jones | Next up: The Valley of Data Access - Part 4 →

The Valley of Data Access - Part 2

2006-06-01T06:22:00.000-07:00

Once I had decided to refactor the data access code for Charlie, the first thing I did was look for a guide. I am frightfully inexperienced with SQL, so I wanted to see if I could find an article, or project, or generator, or anything else, that would guide me.

I started with Google. But every article I found presented the same technique: code a single method that will create a database connection, create a command, add the parameters, open the connection, execute the command, get the data, and close the connection. Yet this everything-in-one-method approach is prone to error and prone to duplication, and is precisely what I wanted to avoid.

I next looked as some open-source projects, including some rather expensive ones, hoping they would take a more considered approach to data access. But no. Each of the projects I looked at took the same everything-in-one-method approach, with an occasional variation being the use of stored procedures in place of hard-coded queries.

I then trialed a commercial code generator. Unfortunately, I could not for the life of me work out how to use it. So I opened a sample project, comprising just four business objects, and used it to generate a data access layer. I don’t know, maybe it’s just me, but I think 3,000 lines of data access code per business object is a tad unnecessary. And then there was the fudgy business object code needed to support the whopping data access code.

After casting about, looking for a guide but failing to find one, I made the increasingly-common decision, “To hell with it, I’ll do it myself.”

I started by looking at the following two pieces of code. The first is the hard-coded query:

String query =
      @"SELECT
            s.Note_Id, 
            Note_CreationDate, 
            Note_UpdateDate, 
            NoteLocalized_Title, 
            NoteLocalized_Content
        FROM
            Charlie_Note s
        INNER JOIN
                Charlie_NoteLocalized sc
            ON
                s.Note_Id = sc.Note_Id
        WHERE
            s.Note_Id = @noteid
        AND
            sc.NoteLocalized_Culture = @culture";

The second was the method that accepts the returned data reader, and populates a business object.

note.CreationDate = Convert.ToDateTime(reader["Note_CreationDate"]);
note.UpdateDate =   Convert.ToDateTime(reader["Note_UpdateDate"]);
note.Title =        Convert.ToString(reader["NoteLocalized_Title"]);
note.Content =      Convert.ToString(reader["NoteLocalized_Content"]);

To put the code into words, the first query string defines the source of the data, while the second method defines the destination of that data. My first thought was, “It would be great if I could create a method that described the mapping between the source table and column names, and the destination property names.” I scribbed down the following code on a piece of paper:

protected Mappings GetMappings()
{
    return new Mappings(
        // Table        // Column            // Property
        "Charlie_Note", "Note_CreationDate", "CreationDate",
        "Charlie_Note", "Note_UpdateDate",   "UpdateDate",
        // and so on
        );
}

My mind then started thinking about how my new data access code would use this mapping information to automate the process of retrieving data from the database and applying it to the business object. I spent a few minutes thinking about this before the little devil on my shoulder whispered, “Alister, you’re talking about creating your own little O/R Mapper here, and we both know you’re not smart enough for that.” The angel on my other shoulder then whispered, “Well, you may be smart enough, but it’s still a dumb idea.”

I went back to looking at the code in my data access methods. I scratched my head a bit. What bothered me is that while each of the methods were very similar, each one had enough little quirks to make it hard to extract any common code. I scratched my head a little more. I then remembered a key design principle from my favourite book for nerds, Head First Design Patterns:

“Identify the aspects of your application that vary
and separate them from what stays the same.”

With that principle in mind, I looked at each of the methods and noted which parts varied and which parts stay the same. In the following listing, the red code is what stays the same within all Retrieve methods.

public override Entity Retrieve(Entity entity, EntityCriteria crit)
{
    Document document = (Document)entity;
    DocumentCriteria criteria = (DocumentCriteria)crit;
    SqlConnection connection =
       new SqlConnection(this.ConnectionString);
    SqlCommand command = new SqlCommand();
    String query =
       @"SELECT
             d.Document_Id,
             d.Document_ParentId,
             d.Document_Name,
             d.Document_FriendlyUrl,
             d.Document_Position,
             d.Document_CreationDate, 
             d.Document_UpdateDate, 
             c.DocumentLocalized_Culture,
             c.DocumentLocalized_Title
         FROM
                Charlie_Document d
             JOIN
                Charlie_DocumentLocalized c
             ON
                c.Document_Id = d.Document_Id
         WHERE ";
    if (criteria.LoadById == true)
    {
       query += " d.Document_Id = @documentid";
       command.Parameters.AddWithValue("@documentid", criteria.Id);
    }
    else if (criteria.LoadByUrl == true)
    {
       query += " d.Document_FriendlyUrl = @friendlyurl";
       command.Parameters.AddWithValue("@friendlyurl", criteria.Url);
    }
    else
    {
       throw new ArgumentException("Invalid criteria.");
    }
    command.CommandText = query;
    command.Connection = connection;
    SqlDataReader reader = null;
    try
    {
       connection.Open();
       reader = command.ExecuteReader();
       if (reader.Read())
       {
          document = NewDocumentFromReader(reader);
       }
       reader.Close();
       base.AddRolesToEntity(document, criteria, connection);
       connection.Close();
    }
    catch (Exception exception)
    {
       throw new DataAccessException(
          "Could not load document.", exception);
    }
    finally
    {
       if (reader != null && reader.IsClosed == false)
          reader.Close();
       if (connection.State != ConnectionState.Closed)
          connection.Close();
    }
    return document;
}

I decided that whatever parts stayed the same would be moved out to a helper class. That would leave the mapper to concentrate on its unique requirements.

What is good about this approach is that by concentrating all of the common code in a single helper class, I would have one point at which to enhance that common code. When I had previously discovered that I was not properly rolling back transactions, I had to go into every one of ten data access classes and make the correction. This way, I would have just one point at which to correct the transaction-based code.

Because this approach had been inspired by the Head First Design Patterns book, my mind started thinking about whether any patterns would help me here. But the little devil on my shoulder whispered in my ear, “Keep it simple, stupid.” The angel on my other shoulder whispered, “You’re not stupid, but keep it simple, sweetheart.”

I decided then that the Mapper class would simply create a query string, create a collection of parameters, and pass them all into the new helper class to be executed. No pattern there, just a clean separation of preparing the database command from executing the database command. Simple.

by Alister Jones | Next up: The Valley of Data Access - Part 3 →

The Valley of Data Access - Part 1

2006-06-01T06:02:00.000-07:00

I am reporting live to you from the valley of data access. All around me I see towering mountains of SQL. Rising in front of me is the DocumentMapper mountain, upon which grows putrid-smelling commands and parameters. To its left is the ContainerMapper mountain, with more foul-smelling stuff. To its right is the RoleMapper mountain, and to the right of that is the UserMapper mountain. I am surrounded by these mountains of SQL.

I hate this place. It is dark and hostile, and I do not have a map or a torch. I need to find a way out. I need to find a way. Out.

I look into my toolkit, and I see three tools at my disposal. One is a grappling hook labelled “consultant,” one is an unopened box labelled “O/R Mapper,” and one is a knife labelled “refactor.”

I’ve thrown the grappling hook to a consultant who stands on top of these mountains. She wraps the hook around a rock, on which she has etched the word “experience,” and I start to climb out. But a man approaches the consultant. He says his name is Shane, and he has something to show her. She must make a choice, between the man dangling at the end of a rope, or the stranger standing before her. She cuts the rope. I fall, crashing back into the valley.

After the pain of the fall subsides, I look at the box labelled “O/R Mapper.” I open it and read the instructions. “For use only by those who know what they’re doing.” That’s not me. I close the box and think, “Maybe some other day.”

I take out the knife labelled “refactor.” I like this knife. I have used it before.

Enough of the story? I thought so too.

Right now, I have ten Mapper classes within Charlie. Each Mapper contains a method for Create, Retrieve, Update, and Delete. Each method is very long, yet each method is not much different from any other. So I have forty large, half-redundant data access methods. Every time I add a feature to Charlie, each method gets a little larger, a little more redundant. Every time I make a change to the database schema, I need to update many, if not all, of the forty separate methods. I need to take control—to reduce the size of the methods and eliminate the redundancy. I have decided to refactor this code before it explodes beyond a maintainable size.

To provide a working sample, here is one of the shortest of the unweildy methods.

public override Entity Retrieve(Entity entity, EntityCriteria crit)
{
    Document document = (Document)entity;
    DocumentCriteria criteria = (DocumentCriteria)crit;
    SqlConnection connection =
       new SqlConnection(this.ConnectionString);
    SqlCommand command = new SqlCommand();
    String query =
       @"SELECT
             d.Document_Id,
             d.Document_ParentId,
             d.Document_Name,
             d.Document_FriendlyUrl,
             d.Document_Position,
             d.Document_CreationDate, 
             d.Document_UpdateDate, 
             c.DocumentLocalized_Culture,
             c.DocumentLocalized_Title
         FROM
                Charlie_Document d
             JOIN
                Charlie_DocumentLocalized c
             ON
                c.Document_Id = d.Document_Id
         WHERE ";
    if (criteria.LoadById == true)
    {
       query += " d.Document_Id = @documentid";
       command.Parameters.AddWithValue("@documentid", criteria.Id);
    }
    else if (criteria.LoadByUrl == true)
    {
       query += " d.Document_FriendlyUrl = @friendlyurl";
       command.Parameters.AddWithValue("@friendlyurl", criteria.Url);
    }
    else
    {
       throw new ArgumentException("Invalid criteria.");
    }
    command.CommandText = query;
    command.Connection = connection;
    SqlDataReader reader = null;
    try
    {
       connection.Open();
       reader = command.ExecuteReader();
       if (reader.Read())
       {
          document = NewDocumentFromReader(reader);
       }
       reader.Close();
       base.AddRolesToEntity(document, criteria, connection);
       connection.Close();
    }
    catch (Exception exception)
    {
       throw new DataAccessException(
          "Could not load document.", exception);
    }
    finally
    {
       if (reader != null && reader.IsClosed == false)
          reader.Close();
       if (connection.State != ConnectionState.Closed)
          connection.Close();
    }
    return document;
}

As I opened by saying, this report is coming to you live. I have printed out a few of these dastardly SQL methods, to look at how I might refactor them. I have not yet started to refactor this code. I’ll update this weblog as I do so.

by Alister Jones | Next up: The Valley of Data Access - Part 2 →

The WebHandler Object

2006-05-27T01:03:00.000-07:00

In the previous weblog entry, I discussed how Charlie used static file handlers to serve existing files. For example, the CssHandler would pick up an existing .css file, maybe update its content, and then return the file.

The next step was to create dynamic file handlers. Whereas the static file handlers are used when there is an existing file to return, the dynamic file handlers are used when there is no existing file to return. Rather, the response needs to be entirely generated from scratch.

This was a relatively simple exercise, since I had a single existing dynamic handler; namely, the PageEngine that used Presenters. I took the existing PageEngine and Presenter objects, and reworked them so that they could present documents in more than just HTML. I will admit that I did a shit job with my earlier attempts to describe the process by which a page is built in Charlie. Let me try again, this time using the new WebHandler object that replaced the previous PageEngine object.

Let’s consider how Charlie handles the following two requests.

/about
/about.rss.xml

The first thing to note is that the same document is being requested (the About page), but the first request is for the document to be in HTML format, while the second request is for the document to be in RSS format.

The first thing Charlie does is hand these requests off to its WebContextBuilder object. The WebContextBuilder looks at the details of the request, and figures out which Document business object is being requested. In both cases, it is the About Document. At this stage, Charlie is unconcerned with the format requested; it is only concerned with which Document is being requested. The requested Document is then loaded up (pulling information from the database or cache) and attached to the current WebContext object.

The second thing that Charlie does is figure out what format has been requested. In these examples, the first request is for HTML format, and the second request is for RSS format. So at this point, Charlie knows which Document business object to present, and in which format to present the document.

The next step is for Charlie to introduce a WebHandler object. The task of this WebHandler is to take the Document business object, and present that Document in the HTML or RSS or other format requested. The WebHandler starts with a blank page that derives from System.Web.UI.Page. In the illustration below, the hand represents the Handler. (That should be easy to remember.)

The first thing the WebHandler does is say to the Document business object, “What template do you want to use?” The Document business object will return the name of the template that the website’s developer has specified. The template will be a simple name such as ‘Silver’ or ‘Playful’. The WebHandler will then add an extension that represents the format in which the document is to be presented. So in the first case .html will be added, while in the second case .rss will be added. The WebHandler then adds the .ascx extension, since templates in Charlie are based on ASP.NET User Controls. The WebHandler loads up the template (such as silver.rss.ascx), and applies it to the blank page.

Within that template may be any number of Placeholder controls that will define where the content is to be placed. For example, a template for an HTML webpage might include Placeholders for the header, content, and footer regions.

At this point, the page is half-constructed, and has slots where the content can go (the Placeholders). The WebHandler then goes back to the Document business object and grabs its collection of Containers. Each Container holds a business object that needs to be presented. For example, one container may hold an Article business object. Another container may hold a Photo business object. But at this point, all the WebHandler does is grab the Containers from the Document business object.

What the WebHandler does next is go through each Container and “tell” it what document format is being presented. So the first Container, which is holding an Article to be presented, knows whether the article is to be presented as HTML or as RSS or as some other format. With that knowledge, the Container says to its Presenter, “Here’s the Article to present, and we need to present it in HTML format.”

The WebHandler then says to the Presenter, “Give me an ASP.NET server control.” If you refer back to the weblog entry on the Presenter object, you’ll see that Charlie calls these server controls Views. So in the example being discussed, the Presenter of the first Container will return an ArticleHtmlView, with the Article business object attached. The second Container is presenting a photo, so its Presenter will return a PhotoHtmlView, with the Photo business object attached.

What this means is that the WebHandler will pull out a single server control, called a View, from each Container.

The final step for the WebHandler is to drop these View server controls onto the templated page. Each Container tells the WebHandler which Placeholder to use, and the position (first, second, third, or later) within that Placeholder.

The WebHandler has now created an ASP.NET Page complete with a Template and a bunch of arranged View server controls. The normal processing of an ASP.NET Page then occurs, and each View server control renders out the appropriate markup. An ArticleHtmlView will render an Article as an HTML element, while an ArticleRssView will render the same Article as an RSS item.

Sigh. I think this weblog entry is just as shit as the earlier entry. Oh well, since I am not being paid to write this weblog, I cannot justify rewriting this entry.

The end of this little story is that by introducing a new WebHandler object, and making the existing Presenter objects accept requests for all manner of different document formats, Charlie is now able to present its documents in multiple formats. What is notable—if I may say so myself—is that this flexibility is now built into Charlie’s architecture. If a client wants his or her pages exposed as Atom feeds, there is no need to write “fudge” code. Rather, by introducing a new ArticleAtomView and a PhotoAtomView, the rest of Charlie remains unchanged (including all existing security and localization).

I have now updated the sample site to present its documents as RSS and Atom feeds. I don’t yet know whether the XML generated is valid, because this is just a proof of concept. But as a proof of concept, I feel that it illustrates Charlie’s flexibility.

by Alister Jones | Next up: The Valley of Data Access - Part 1 →

Static File Handlers

2006-05-21T07:31:00.000-07:00

In the second of the four weblog entries concerning Charlie’s cool URLs, I noted that Charlie uses a number of custom file handlers. The examples provided were a CSS Handler, a JavaScript Handler, a GIF Handler, and a JPEG Handler.

If you know a little bit about HTTP responses, you will know that there are three common response types. If the response is valid, it is a 200 OK response. If the file could not be found, it is a 404 Not Found response. If the file is found, but it has not been modified since the browser last requested it, it is a 304 Not Modified response. So that I did not have to duplicate this logic in each of my separate file handlers, I started by creating a base StaticHandler class. Here is the code.

using System;
using System.IO;
using System.Web;
using System.Security;

namespace Charlie.Framework.Interface.Handlers
{
   public abstract class StaticHandler : IHttpHandler
   {
   
      // Members required by IHttpHandler interface
   
      public void ProcessRequest(HttpContext context)
      {
         FileInfo fileInfo = new FileInfo(context.Request.PhysicalPath);
         if (IsFileValid(fileInfo) == false)
         {
            ReturnNotFoundResponse();
         }
         else if (IsFileModified(fileInfo, context) == false)
         {
            ReturnNotModifiedResponse(context);
         }
         else
         {
            ReturnFileResponse(fileInfo, context);
         }
      }
      
      public Boolean IsReusable
      {
         get
         {
            return true;
         }
      }
      
      // File Tests
      
      private Boolean IsFileModified(FileInfo fileInfo, HttpContext context)
      {
         Boolean isModified = true;
         String modifiedSince = context.Request.Headers["If-Modified-Since"];
         if (modifiedSince != null)
         {
            try
            {
               DateTime lastModified = fileInfo.LastAccessTimeUtc;
               DateTime lastReceived = Convert.ToDateTime(modifiedSince);
               if (lastModified == lastReceived)
               {
                  isModified = false;
               }
            }
            catch
            {
            }
         }
         return isModified;
      }

      private Boolean IsFileValid(FileInfo fileInfo)
      {
         if (File.Exists(fileInfo.FullName))
         {
            return true;
         }
         return false;
      }      
      
      // Responses

      private void ReturnNotFoundResponse()
      {
         throw new HttpException(404, "File not found.");
      }

      private void ReturnNotModifiedResponse(HttpContext context)
      {
         context.Response.StatusCode = 304;
         context.Response.SuppressContent = true;
      }
      
      private void ReturnFileResponse(FileInfo fileInfo, HttpContext context)
      {
         try
         {
            HttpResponse response = context.Response;
            this.ReturnFile(fileInfo, response);
            response.Cache.SetLastModified(fileInfo.LastWriteTime);
            response.Cache.SetCacheability(this.Cacheability);
            response.Cache.SetExpires(this.CacheExpiry);
         }
         catch (SecurityException)
         {
            throw new HttpException(401, "Access to file denied.");
         }
         catch
         {
            // it is more secure to not signfiy an error
            throw new HttpException(404, "Unable to serve file.");
         }
      }
      
      // Default Cache Values

      protected virtual HttpCacheability Cacheability
      {
         get
         {
            return HttpCacheability.Public;
         }
      }

      protected virtual DateTime CacheExpiry
      {
         get
         {
            return DateTime.Now.AddDays(1);
         }
      }
      
      // Abstract Member
      
      protected abstract void ReturnFile
            (FileInfo fileInfo, HttpResponse response);
   }
}

The code is very simple, and parts of it come from Milan Negovan’s Adding Variables To Style Sheets article. There is a single abstract ReturnFile method which a concrete handler class must implement. By creating this base class that forces a concrete handler to implement just one method, let’s see just how simple those handlers can be.

Here is the code for Charlie’s CSS Handler.

using System;
using System.IO;
using System.Web;

namespace Charlie.Framework.Interface.Handlers
{
   public class CssHandler : StaticHandler
   {
      protected override void ReturnFile
            (FileInfo fileInfo, HttpResponse response)
      {
         StreamReader reader = fileInfo.OpenText();
         String css = reader.ReadToEnd();
         reader.Close();
         response.Write(css);
         response.ContentType = "text/css";
      }
   }
}

Can’t get much simpler than that, can you?

Here is the code for Charlie’s JavaScript Handler.

using System;
using System.IO;
using System.Web;

namespace Charlie.Framework.Interface.Handlers
{
   public class JavaScriptHandler : StaticHandler
   {
      protected override void ReturnFile
            (FileInfo fileInfo, HttpResponse response)
      {
         StreamReader reader = fileInfo.OpenText();
         String script = reader.ReadToEnd();
         reader.Close();
         response.Write(script);
         response.ContentType = "text/javascript";
      }
   }
}

And here is the code for Charlie’s GIF Handler:

using System;
using System.IO;
using System.Web;
using System.Drawing;
using System.Drawing.Imaging;

namespace Charlie.Framework.Interface.Handlers
{
   public class GifHandler : StaticHandler
   {
      protected override void ReturnFile
            (FileInfo fileInfo, HttpResponse response)
      {
         Bitmap bitmap = new Bitmap(fileInfo.FullName);
         bitmap.Save(response.OutputStream, ImageFormat.Gif);
         bitmap.Dispose();
         response.ContentType = "image/gif";
      }
   }
}

The advanced developers in the audience will be thinking to themselves, “Well that’s fine, Alister, but the built-in StaticFileHandler would have done the same for you.” That’s true, but only because I have not yet updated the handlers to do anything special. But as a test, I updated the JPEG Handler to stamp a copyright notice on any returned JPEG image. Here is the code.

using System;
using System.IO;
using System.Web;
using System.Drawing;
using System.Drawing.Imaging;

namespace Charlie.Framework.Interface.Handlers
{
   public class JpegHandler : StaticHandler
   {
      protected override void ReturnFile
            (FileInfo fileInfo, HttpResponse response)
      {
         Bitmap bitmap = new Bitmap(fileInfo.FullName);

         // add copyright notice
         Font font = new Font("Verdana", 10);
         Brush brush = new SolidBrush(Color.Black);
         String copyright = "Copyright";
         Graphics canvas = Graphics.FromImage(bitmap);
         canvas.DrawString(copyright, font, brush, 0, 0);

         bitmap.Save(response.OutputStream, ImageFormat.Jpeg);
         bitmap.Dispose();
         response.ContentType = "image/jpg";
      }
   }
}

As you will see, the concrete handler classes are about as simple as you could want. Yet, they allow Charlie to have full control over every file used in a website. For example, tokens can be placed within a CSS style sheet (such as “[[CorporateColour]]”) and then used in a simple find-and-replace operation before the CSS file is returned. Another example was provided above, where a copyright notice is stamped on the JPEG image returned.

The only other thing to show is the code needed in the Web.config file that points the incoming file requests to the appropriate handler. To minimise the width of the code, I have replaced the real namespace with the word “Namespace”.

<system.web>
   <httpHandlers>
      <add verb="*" path="*.asxx" 
           type="Namespace.ViewHandler, Charlie.Framework"/>
      <add verb="GET" path="*.js" 
           type="Namespace.JavaScriptHandler, Charlie.Framework"/>
      <add verb="GET" path="*.css" 
           type="Namespace.CssHandler, Charlie.Framework"/>
      <add verb="GET" path="*.gif" 
           type="Namespace.GifHandler, Charlie.Framework"/>
      <add verb="GET" path="*.jpg" 
           type="Namespace.JpegHandler, Charlie.Framework"/>
      <add verb="GET" path="*.jpeg" 
           type="Namespace.JpegHandler, Charlie.Framework"/>
   </httpHandlers>
</system.web>

If you visit the little sample site that I have put online, keep in mind that Charlie is serving every single file used in that site—not just the two pages that are currently viewable. And if you want to do the same on your own website, you now have the code.

by Alister Jones | Next up: The WebHandler Object →

Charlie’s Birth

2006-05-20T23:23:00.000-07:00

Charlie has been born. Like any new-born baby, he is far from fully developed. Still, he is now out there in the world, ready to grow, ready to learn, ready to make friends.

You may see him at www.edition3.net.

Right now there’s not too much to see, and what there is to see is very basic. The HTML and CSS came from my first foray into CSS-based design, some two years ago. The content comes from Wikipedia. In other words, what you will see there is nonsense content, because I am still working on beneath-the-skin stuff.

So have a play with the baby, and let me know if he poops or cries.

by Alister Jones | Next up: Static File Handlers →

Think Big

2006-05-18T18:49:00.000-07:00

I made a mistake by thinking small. But I learned a lesson that I would like to share.

I was looking at the feature set for Charlie, and I came to the following item:

allow for different document types (webpage, PDF, Word, RSS, etc.)

It occurred to me that I should get this flexibility in place. Otherwise, I would flesh out Charlie’s ability to serve webpages, and then have to retrofit the flexibility to serve other types of responses. And if there is one thing I have learned so far, it is to get flexibility in place as early as possible.

The only article I have ever read on serving different response types is One Site, Many Faces. It is a great article, wherein the author details how you can have one HttpHandler for each response type that your website is to serve. So, you’d have one TextHandler, one RssHandler, and so on. But the article does have a notable limitation. The limitation is perfectly acceptable, since it kept the article to a reasonable length. But the limitation would have manifested itself in Charlie. The limitation in the article is that each handler knows exactly what business object it is to present, which is an Article. The Article business object looks roughly like this:

public class Article
{
    public String Title;
    public String Body;
    public String Writer;
}

Because each handler knows exactly what business object it is to serve, the code to do so is very simple. Here is a snippet of how the TextHandler might look:

public class TextHandler
{
    public void PrepareResponse(Article article)
    {
        String response;
        response  = article.Title + NewLine;
        response += article.Writer + NewLine;
        response += article.Body;
        SendResponse(response);
    }
}

And here is a snippet of how the RssHandler might look:

public class RssHandler
{
    public void PrepareResponse(Article article)
    {
        String response;
        response  = "<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>";
        response += "<rss>";
        response += "   <title>" + article.Title + "</title>";
        response += "   <author>" + article.Writer + "</author>";
        response += "   <description>" + article.Body + "</description>";
        response += "</rss>";
        SendResponse(response);
    }
}

So, by working with a single, known business object (an Article), each of the separate Handlers were kept simple, and the website is able to serve the same article in many different ways.

The problem I faced was how to overcome this limitation of a single, known business object? To illustrate the problem, we can consider just two separate business objects, an Article and a Weblog.

public class Article            public class Weblog
{                               {
    public String Title;            public String Subject;
    public String Body;             public String Entry;
    public String Writer;           public Int32  BloggerId;
}                               }

What you will see is that the two separate business objects do not share a single property. So, the above TextHandler and RssHandler could not work with a Weblog business object. In theory I could have introduced a new set of handlers for every new business object, so that I’d end up with an ArticleTextHandler and ArticleRssHandler, and a WeblogTextHandler and WeblogRssHandler. That would work, but that’s not flexibility, that’complexity. Instead, it would more flexible if I introduced an interface that defined what sort of object could be presented by the TextHandler, and what sort of object could be presented by the RssHandler. Looking just at the RssHandler, here is how that interface might look.

public interface IRssItem
{
    String Title;
    String Author;
    String Description;
}

With that interface in place, we can update the RssHandler to look like this:

public class RssHandler
{
    public void PrepareResponse(IRssItem item)
    {
        String response;
        response  = "<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>";
        response += "<rss>";
        response += "   <title>" + item.Title + "</title>";
        response += "   <author>" + item.Author + "</author>";
        response += "   <description>"+item.Description+"</description>";
        response += "</rss>";
        SendResponse(response);
    }
}

This interface means that the RssHandler can now work with any business object that implements the IRssItem interface. By using interfaces this way, I could give Charlie one TextHandler, one RssHandler, one AtomHandler, and so on. And these handlers could work with any business object, so long as that business object implemented the required interface. This was the sort of flexibility I was looking for.

But here is where I made a mistake in my approach to this problem. By looking at and thinking about the code I have just shown you, my thoughts were down at the “small” details. Namely, I started thinking about how I could update the business objects so that they each presented the same IRssItem interface. Here again are the business objects in question:

public class Article            public class Weblog
{                               {
    public String Title;            public String Subject;
    public String Body;             public String Entry;
    public String Writer;           public Int32  BloggerId;
}                               }

The straight forward approach would be to implement that interface directly in each business object:

public class Article            public class Weblog
    : IRssItem                      : IRssItem 
{                               {
    public String Title;            public String Subject;
    public String Body;             public String Entry;
    public String Writer;           public Int32  BloggerId;
    
    IRssItem.Title                  IRssItem.Title
      { return this.Title }           { return this.Subject }
    IRssItem.Author                 IRssItem.Author
      { return this.Writer }          { return GetName(this.BloggerId) }
    IRssItem.Description            IRssItem.Description
      { return this.Body }            { return this.Entry }
}                               }

I only had to think about that for a few seconds before I realized that that was not an option at all. Each time I introduced a new handler and interface, I’d have to go in and change every business object that wanted to be presented by that handler. That too is not flexibility, that’s complexity.

I will admit that I banged about for nearly four hours trying to think of different solutions. The reason that I struggled for so long was that I was thinking about the small details. It was only when I went for a drive (I’ve crashed my motorbike) that I started thinking big, and a solution occurred to me.

Because I was driving and could not write down the code for a business object, I visualised each business object as a box. And because the Article business object and the Weblog business object have different properties, they expose different interfaces.

I then pictured the RssHandler as being a box, which has a defined IRssItem interface for any business object that it is to display.

The problem to be solved then was how can vastly different business objects, each exposing a different interface, be made acceptable to the RssHandler which requires each business object to expose the defined IRssItem Interface? The following picture formed in my mind:

Once that picture formed in my mind, the solution became obvious. I needed an adapter from the Adapter Pattern.

By using an adapter, I never needed to change the business objects in any way whatsoever. Even better, a given business object could be made to plug into any different type of handler. Putting the Weblog business object to one side, let’s see how the Article business object can be made to work with any and every handler, including its own specialised ArticleHandler, again without having to change the business object in any way.

So, using the code examples above, what would the ArticleRssAdapter look like?

public class ArticleRssAdapter : IRssItem
{
    public ArticleRssAdapter(Article article)
    {
        this.article = article;
    }
    private Article article;
    
    public String IRssItem.Title
    {
        get 
        { 
            return article.Title; 
        }
    }

    public String IRssItem.Author
    {
        get 
        {
            return article.Writer; 
        }
    }

    public String IRssItem.Description
    {
        get 
        { 
            return article.Body; 
        }
    }
}

You will see that this code is extremely simple. But even though it is simple, the use of adapters means that any business object can be made to work with any handler object. This is infinitely flexible.

The end solution is so simple that I really should have seen it. But, I was thinking small, and looking at the code. I was not thinking big, and looking at the objects. So that’s the lesson for today kids: think big.

by Alister Jones | Next up: Charlie’s Birth →

Story Time

2006-05-17T17:04:00.000-07:00

I have been told by an experienced developer that my writing sucks and my explanations are incomprehensible. Judging by the lack of responses I receive to my weblog entries, it would seem that she is right. I enjoy maintaining this weblog, but I don’t want to think that I’m just babbling to myself. I need to find a writing style that is more interesting and more accessible.

To that end, I would like to point you to a reply that I have just now submitted to the ASP.NET Forums, where I use the moniker ‘SomeNewKid’. Is its story style any better? I don’t know how successful the story style would be in the describing the life of Charlie, but I’m willing to give it a go.

Please, let me know how I can make the life of Charlie more interesting. If you’d rather tell me privately, please feel free to contact me. But do please tell me.

by Alister Jones | Next up: Think Big →

Charlie has Cool URLs - Part 4

2006-05-13T10:42:00.000-07:00

I lied to you earlier. It was a white lie, with no harm intended. But it was a lie nonetheless, and for that I’m sorry. If I explain, will you forgive me?

In my first and second look at Charlie’s URLs, I said that Charlie uses a slight-of-hand to add the .aspx extension to any incoming URL that has no extension, and that the Web.config file has been updated so that any request with an .aspx extension gets handled by the PageBuilder class. This process is indeed what happens, but it is not an .aspx extension that Charlie uses to route page requests to its PageBuilder class. Rather, it is a custom .asxx extension, because I wanted to leave the built-in .aspx extension untouched. Let’s see why.

Toward the end of my second weblog entry on cool URLs, we saw the following diagram.

Consider that a website is presenting a weblog entry, and within that weblog entry is a photo of Pamela Anderson before she was attacked by a bicycle pump. The URL for that photo may be the following:

/weblog/2006/apr/14/a-rare-photo-indeed.jpg

If you look at the diagram above, you will see that the request will be handled by Charlie’s own JPEG Handler. The JPEG Handler will just pick up the .jpg file and return it, which is all we want it to do.

Consider now a website that is presenting a photographer’s photo gallery, and within that gallery is a photo of the exquisite Michelle Pfeiffer. The URL for that photo may be the following:

/photos/celebrities/michelle-pfeiffer/batman-premiere.jpg

If you recall that Charlie uses a fall-through approach of matching URLs with plugins, you will see that the above URL may be associated with the PhotoGallery plugin. And here is where it gets interesting.

Most plugins do not care about requests for GIF images and JPEG photos, so they will not want to handle such requests. These plugins will simply let the request go through to the Charlie’s own designated handler (see the diagram above).

Some plugins, however, do care about requests for GIF images and JPEG photos. There plugins will want to handle such requests themselves, rather than allowing the request to go through to Charlie’s own default handlers. The PhotoGallery plugin, for example, may wish to present the .jpg image not by itself, but on a page with an attractive border, a title, and a copyright notice. The PhotoGallery plugin may also wish to record how many times a particular photo has been viewed. Put simply, the PhotoGallery plugin should be able to do whatever it likes with requests for photos.

So, how can we update Charlie so that a plugin can take control of a request if it wants control, but otherwise allow a request to flow through to the default handlers? If we look at again at the diagram above, we see an ideal point at which we can do this.

At the point indicated, all authentication and authorization checks have been performed, but the handler has not yet been chosen. This is the point at which Charlie says to the Plugin associated with the request, “Hey buddy, if you want to handle this request yourself, tell me now, otherwise I’ll handle it myself.” The way Charlie does this is by giving the associated Plugin the current WebContext object, which allows the Plugin to do whatever it likes to the HttpContext, HttpRequest, and HttpResponse objects it holds. The PhotoGallery plugin can therefore “catch” any JPEG requests, and re-route them to its own handler. Here is a simple example:

public void FilterWebContext(WebContext webContext)
{
    if (webContext.WebRequest.Address.Extension == "jpg")
    {
        String handler = "~/plugins/photogallery/jpgHandler.aspx";
        webContext.HttpContext.RewritePath(handler);
    }
}

There’s quite a bit wrong with the above code example, but it gives the basic idea of how Charlie allows its own plugins to update the above diagram in the following way.

This explains why I wanted to leave the .aspx extension untouched. If a plugin wishes, it can redirect an incoming request to one of its own .aspx pages, in which case ASP.NET’s own PageHandlerFactory takes control of loading up the .aspx page and any code-behind file. Charlie’s own PageBuilder class is associated only with the custom .asxx extension that Charlie adds to any request that does not otherwise have an extension.

What was particularly nice here is that updating Charlie to allow plugins to “take control” of an incoming request was a further example of emergence. The existing Web System and Plugin System presented the fertile soil from which this new feature could be grown. No part of Charlie’s architecture or design had to be changed or fudged in any way. The solution emerged naturally from its existing architecture and existing design. I am not patting myself on the back here. I am sharing the lesson that I am learning again and again: a little bit of object-oriented design goes a long, long way.

This weblog entry concludes the look at Charlie’s cool URLs, and how those URLs tie in with Charlie and its plugins.

by Alister Jones | Next up: Story Time →

Charlie has Cool URLs - Part 3

2006-05-13T09:03:00.000-07:00

Let’s presume that Charlie is serving a photographer’s website. The website comprises an extensive photo gallery, the photographer’s weblog, an about page, and a contact page. Let’s look at some of the URLs that might apply to the photo gallery.

www.example.com/photos

www.example.com/photos/landscapes
www.example.com/photos/weddings

www.example.com/photos/weddings/susan-and-mark
www.example.com/photos/weddings/susan-and-mark/kiss
www.example.com/photos/weddings/susan-and-mark/rings

All of these pages will be handled by Charlie’s PhotoGallery plugin. You and I can see that quite clearly, since the URLs are all children of the parent /photos URL. But how can Charlie know that all of these URLs are to be handled by Charlie’s PhotoGallery plugin? There are at least three ways.

The first way would be for Charlie’s database to keep a record of every single URL and the Plugin associated with the URL. That’s not as easy as it sounds, because many of the URLs might be generated on-the-fly. For example, if the visitor is looking at the photo of Susan’s ring, there may be a link to view a closeup of the photo. Clicking on the link might navigate to the following URL:

www.example.com/photos/weddings/susan-and-mark/rings/closeup

The PhotoGallery plugin may use lots of these on-the-fly URLs, and it would be extremely difficult if Charlie had to record every single possible URL. This is not really a viable approach.

The second way for Charlie to associate URLs with Plugins is to use a wildcard system. For example, Charlie’s database could record the following single URL, and associate this record with the PhotoGallery plugin.

/photos/*

To state the obvious, any child of the /photos URL will be handled by the PhotoGallery plugin. This approach accommodates any of the on-the-fly URLs created by the PhotoGallery plugin, so this second approach is better than the first. However, it does carry a limitation. What if we don't want every single child of /photos to be handled by the PhotoGallery plugin? We may want /photos/order to be handled by the Payment plugin, and /photos/copyright to be handled by the SimpleHtml plugin.

In terms of code, we could use a system of regular expressions in order to perform our wildcard matches. By using regular expressions, the wildcard search can “exclude” certain URLs (such as /photos/order and /photos/copyright) from the wildcard test (such as /photos/*). However, that is getting very finicky, and is not a very flexible or sustainable approach.

The third way for Charlie to associate URLs with Plugins is to use a fall-through system. First, Charlie maintains a record of parent URLs. These are not necessarily actual URLs like in the first option, and are not wildcard URLs like in the second option, but are a record of parent URLs and the associated Plugin. Like this:

/photos              - PhotoGallery plugin
/photos/order        - Payment plugin
/photos/copyright    - SimpleHtml plugin

Charlie looks at the requested URL, which may be the following:

/photos/weddings/susan-and-mark

Charlie checks whether this URL has a match in its list of parent URLs. It does not, so Charlie lops off the last part of the URL, to be left with this:

/photos/weddings

Charlie checks whether this URL has a match in its list of parent URLs. It does not, so Charlie now lops off more of the original URL, to be left with this:

/photos

Now a match will be found, and Charlie knows to hand this request off to its PhotoGallery plugin. Even more helpful is that anything that was lopped off the original URL is now considered a parameter. When the PhotoGallery plugin receives this request, it will be split like this:

document = /photos
parameter = /weddings/susan-and-mark

The PhotoGallery plugin can look at this parameter to know exactly what it needs to display.

Now let’s consider a different URL.

/photos/copyright

Charlie checks whether this URL has a match in its list of parent URLs. It does, and it is associated with the SimpleHtml plugin.

So this third approach keeps lopping off the ends of the incoming URL until a parent-URL match is found. This is just as flexible as the second, wildcard approach, but overcomes the limitation of that second approach, which was how to exclude certain URLs from its wildcard matches. This third approach would be awkward if each test required a trip to the database. However, Charlie loads up a DocumentMap (like a site map) for each Domain it serves, and places this DocumentMap in its high-priority cache. These fall-through checks therefore incur little performance penalty, but provide great flexibility in associating URLs with Plugins.

Need I state that it is the third approach that has been adopted for Charlie?

by Alister Jones | Next up: Charlie has Cool URLs - Part 4 →

Charlie has Cool URLs - Part 2

2006-05-12T19:08:00.000-07:00

In my previous weblog entry, I said that it has been a four-step process by which I’ve given Charlie cool URLs. The first step was to implement URL rewriting, and was the topic of the previous entry. The second step was to remove any extensions from URLs, and that’s what I’ll talk about here.

Let’s look an one of Apple’s cool URLs:

www.apple.com/ipod

What is significant about this URL is that it contains no extension. The page name is just ipod, and not ipod.htm or ipod.aspx. Internet Information Services isn’t too keen on such URLs. If IIS receives a request for ipod.asp, it passes the request off to ASP for processing. If IIS receives a request for ipod.aspx, it passes the request off to ASP.NET for processing. If IIS receives a request for ipod.gif, it handles the request itself by simply returning the requested GIF image. But, by default, IIS does not know what to do with requests that have no extension. (In reality, IIS “tests” certain default documents, but that’s a bit beside the point here.)

But there is a trick to get IIS to pass to ASP.NET any request for a file with no extension. The trick is to use what is known as a wildcard mapping (.*) in IIS that points to ASP.NET. This wildcard mapping says to IIS, “Whatever request you get, no matter what its extension, or even if it has no extension, pass that request on to ASP.NET.”

So with this wildcard mapping, cool URLs with no extension can be used in an ASP.NET website. But there is a catch. The wildcard mapping has told IIS to pass all requests to ASP.NET. That means the ASP.NET application will handle requests for style.css, script.js, image.gif, photo.jpg, and so on. For the most part, this presents no problem. ASP.NET will use its StaticFileHandler to simply pick up the requested file and send it back, which is precisely what IIS would have done anyway. But even requests for image.gif and photo.jpg will be subject to the same processing as for page.aspx, including authentication and authorization checks. Moreover, if your application is logging requests, you’ll be logging every request for every little file.

If you’re aware of this catch, then it also presents an opportunity. The ASP.NET application can now adjust those image or text files before being returned, or can create them from scratch. If your application presents a photographer’s photos, the application can stamp a copyright notice on every returned JPEG file. This is an opportunity that Charlie seizes, but I’ll come back to it in a moment.

There is another gotcha about using URLs with no extension. Previously, it was IIS that didn’t know what to do with a request for a file named ipod with no extension. We solved this by giving IIS a wildcard mapping that tells it to pass all requests onto ASP.NET. But this just passes onto the ASP.NET application the same problem. What should it do with a file named simply ipod?

Rather than let ASP.NET deal with this problem, Charlie will add the .aspx extension to any incoming request that does not have an extension. So, while the visitor sees /ipod in his or her browser, Charlie changes this to be /ipod.aspx before full processing of the request occurs. In the previous weblog entry, I described how a new line in the Web.config file told ASP.NET that all requests for an .aspx page should go to the PageBuilder class. So that is the process by which the user sees cool URLs with no extensions, but ASP.NET sees the .aspx extensions that it knows and loves. Charlie just performs a little slight of hand, adding the .aspx extension when ASP.NET is not looking, and thereby gets a cool URL such as /ipod to be handled by its PageBuilder class.

Getting back to the opportunity that presents itself with a wildcard mapping in IIS, Charlie must handle all requests for all file types. To save having to write a thousand words, I’ll draw a diagram that shows what happens, by default, to five separate requests to the Charlie application. Because IIS has been instructed to pass all requests through to ASP.NET, Charlie will need to handle the requests for page.aspx, style.css, script.js, image.gif, and photo.jpg. By default, the last four will be handled by ASP.NET’s built-in StaticFileHandler.

But Charlie would like to seize the opportunity that presents itself. Namely, Charlie can implement its own way of handling requests for CSS Stylesheets, JavaScript files, GIF images, and JPEG photos. That way, Charlie can elect what to do. Charlie can just pick up the requested file and return it, just as the StaticFileHandler would do. Or, Charlie can pick up the requested file and adjust it (such as adding a copyright notice) before returning it. Or, Charlie can dynamically create the file requested (such as creating a GIF graph). ASP.NET makes this easy. Here are the instructions we write in the Web.config file, followed by a diagram showing the result.

<httpHandlers>
    <add verb="*" path="*.aspx" type="PageHandler, AssemblyName"/>
    <add verb="*" path="*.css"  type="CssHandler,  AssemblyName"/>
    <add verb="*" path="*.js"   type="JSHandler,   AssemblyName"/>
    <add verb="*" path="*.gif"  type="GifHandler,  AssemblyName"/>
    <add verb="*" path="*.jpg"  type="JpegHandler, AssemblyName"/>
</httpHandlers>

As I write this, I have not yet created the extra handlers. However, my little alisterjones.com website has these extra handlers, so it will be a simple matter. I’ll write about the handlers when I add them to Charlie.

So there you have a description of how Charlie’s URLs are so cool they don't even have extensions in them, and how Charlie is able to handle all requests for all files for all websites it serves.

by Alister Jones | Next up: Charlie has Cool URLs - Part 3 →

Charlie has Cool URLs - Part 1

2006-05-12T07:39:00.000-07:00

An ongoing joke in Beavis and Butthead is that, to the boys, everything is either cool or it sucks. There is no inbetween. It is cool. Or it sucks.

I have the same black-and-white approach to URLs. A given URL is either cool, or it sucks. Apple has cool URLs:

www.apple.com/ipod

The URLs for Creative suck:

www.creative.com/products/product.asp?category=213&subcategory=214&product=11519

Charlie has cool URLs, if I do say so myself. It has been a four-step process to give Charlie cool URLs, and I’ll dedicate one weblog entry to each step.

The first step to giving Charlie cool URLs was to implement URL rewriting. This is the process by which the incoming cool URL (such as /ipod) is rewritten to the “real” file that handles the request (which may be /products.aspx?id=10). A common technique is described in URL Rewriting in ASP.NET, whereby all incoming page requests are rewritten to one of a handful of actual .aspx pages. The code looks like this:

String coolUrl = HttpContext.Current.Request.Path;
String targetUrl = UrlRewriter.GetHandler(coolUrl);
HttpContext.Current.RewritePath(targetUrl);

In such a URL rewriting system, the developer maintains a handful of .aspx pages, such as ShowProduct.aspx, ShowCategory.aspx, and ShowPerson.aspx. The details of what to show are passed in via QueryString variables. So, the above targetUrl string might look something like this:

ShowProduct.aspx?category=16&product=128&view=details

For most web applications, the process of sheperding all incoming requests to one of a handful of .aspx pages is all that is needed. But Charlie has a special requirement that means this common approach isn’t quite right for my project. The special requirement is that Charlie must host multiple websites from a single installation, and each website must be able to generates pages that have nothing at all in common with pages from another website. For this requirement to be met, the pages must be completely dynamic in their creation. With an .aspx page, some of it will be static (whatever is already on the .aspx page) and some of it will be dynamic. So having a handful of .aspx pages would not meet this requirement of Charlie. So, what are the alternatives? Well, there are two least two.

The first option available for a web application to dynamically create the entire page is to have a single .aspx page with nothing other than a Page directive that points to a page-building code-behind file:

<%@ Page language="C#" AutoEventWireup="false" Inherits="PageBuilder" %>

The page has nothing on it, so its entire content must be generated dynamically. If we place this .aspx in the root folder of the web application, the above URL rewriting code would be adjusted to look something like the following:

String coolUrl = HttpContext.Current.Request.Path;
String querystring = UrlRewriter.GetQueryString(coolUrl);
HttpContext.Current.RewritePath("~/default.aspx?" + querystring);

The QueryString might look something like this:

page=36&template=blue&category=16&product=128&view=details

So, whereas the technique from the article means the developer maintains a handful of separate .aspx pages, this first alternative means the developer has just a single .aspx that, frankly, does nothing—the work is done by the page-building code-behind.

The second option available for a web application to dynamically create the entire page is a variation of the first option just described. Whereas the first option used a single .aspx and a single code-behind, the second option does away with the .aspx page altogether. The .aspx page does nothing except point to the code-behind, yet there’s another way we can point to the code-behind. Namely, we can use the Web.config file to direct all .aspx page requests to that code-behind. Here is the relevant section of the Web.config file:

<httpHandlers>
    <add verb="*" 
         path="*.aspx" 
         type="PageBuilder, AssemblyName"/>
</httpHandlers>

What this means is that any request with an .aspx extension will be handled by the PageBuilder class. There is no need for any .aspx page to exist, which means that the above URL rewriting code can be modified like this:

String coolUrl = HttpContext.Current.Request.Path;
String targetUrl = UrlRewriter.GetHandler(coolUrl);
HttpContext.Current.RewritePath(targetUrl);

Yes, this is the same code as that shown first. However, previously the targetUrl had to point to an existing .aspx page. This time, the targetUrl does not need to point to any real .aspx page, and could look like this:

show-a-porsche-with-a-black-background.aspx?otherwise=ferrari

Using a virtually-empty .aspx file and using a Web.config entry both do the same thing: point to the code-behind that dynamically creates the entire page. While I could have used either option, I went with the second option of using the Web.config to point to the code-behind. First, I didn’t fancy having a do-nothing .aspx page. Second, and for reasons I’ll come to in my next weblog entry, Charlie would end up needing quite a number of these do-nothing pages. The Web.config option makes the do-nothing pages unnecessary, so that is the approach I took.

by Alister Jones | Next up: Charlie has Cool URLs - Part 2 →

The Two Devils of SQL

2006-05-09T09:14:00.000-07:00

Charlie and I have a real love-hate relationship with SQL Server. Charlie loves how fast and how flexible it is. I hate how hard it is to use. I have been using ASP.NET for about three years now, and in that time I have tried to install innumerable free and commercial ASP.NET applications. I have succeeded only about ten percent of the time, with the problem always being the database. This triggered for a me a deep dislike for SQL Server.

When I first started thinking about creating a website framework, I was very tempted to use XML files as the data store. I like and understand XML, whereas I did not like and did not understand SQL Server. Fortunately, one company and one person came to my aid, and made SQL Server a viable data store for Charlie.

The company that came to my aid was Microsoft. Last year Microsoft deemed me worthy of an MVP award, and with that award came a subscription to the MSDN Network. I was able to download SQL Server 2000 and, with its Enterprise Manager, finally have a user interface to the database. The interface sucks, but at least I finally had one.

The person that came to my aid was Terri Morton. Poor Terri had to endure a hundred questions—and as many expletives—while I fumbled about with installing, configuring, and finally using SQL Server. Fortunately, Terri is both smarter and more patient than I am, and stuck with me until I finally had the SQL beast under control. Thank you, Terri.

So Charlie was undertaken with SQL Server as its data store. While I can get SQL Server to do what I need, I still dislike working with the database. The easy stuff, such as writing CRUD methods, is so boring and so repetitive that it begs to be automated. The hard stuff, such as schema design and stored procedures, is so difficult for me that I know I’ll never do a good job with it. With the hard stuff, I have taken a pragmatic approach that would surely please any seasoned architect: I’m just not going to worry about it. If I can get Charlie working with a dirt-simple database design and no stored procedures, then that is good enough. If I ever get the darn thing finished, I’ll engage a database person to come in and rework the database for Charlie version 2.0. Until that time, if my dirt-simple database works, then that’s good enough.

It is the easy stuff that concerns me. Writing CRUD methods is so boring and so repetitive that I have given serious consideration to using the WilsonORMapper. This leads me to a choice between two devils. The devil I know is hand-coded CRUD methods. The devil I don’t know is Object-Relational Mapping.

Taking the advice of the aardvark—that simple is better than complicated—I have decided to stick with the devil I know, and just hand-code all CRUD methods. Sure it’s boring, sure it’s repetitive, but it is also simple and flexible. I have considered refactoring the database code, so that some of the repetitive code can be moved to either a base class or a utility class. However, I have decided that I will just keep the code simple, if repetitive. If my yet-to-be-engaged database specialist wants to introduce changes, that will be his or her prerogative.

I have however decided to give a friendly wink to the devil I don’t know. One of the nice features of Paul Wilson’s ORMapper is that it supports an IObjectHelper interface. Without this interface, object-relational mapping is performed by reflection, which is a relatively slow process. With this interface, the object-relational mapping is performed through a known indexer:

public interface IObjectHelper
{
    Object this[String memberName] { get; set; }
}

Here is a very simple business object that implements this IObjectHelper interface:

public class Person : IObjectHelper
{
    public Int32 ID
    {
        get
        {
            return this.id;
        }
    }
    private Int32 id;
    
    public String Name
    {
        get
        {
            return this.name;
        }
        set
        {
            this.name = value;
        }
    }
    private String name;
    
    public Object this[String memberName] 
    {
        get 
        {
            switch (memberName) 
            {
                case "id": return this.id;
                case "name": return this.name;
                default: throw new ArgumentException
                         ("Invalid Member", memberName);
            }
        }
        set
        {
            switch (memberName) 
            {
                case "id": this.id = (Int32)value; break;
                case "name": this.name = (String)value; break;
                default: throw new ArgumentException
                         ("Invalid Member", memberName);
            }
        }
    }
}

There are two great features here. First, the IObjectHelper interface is optional. If a business object does not implement this interface, then Paul’s WilsonORMapper will simply use reflection to populate the business object. If a business object does implement this interface, then the WilsonORMapper will use it to avoid the costs of reflection. You can read more about this in Paul’s weblog entry on O/R Mappers: Avoiding Reflection. The second great feature is that the interface provides a single point at which the Persistence layer interacts with the Business layer. This is of more benefit than is immediately apparent. To see why, have a look at a property of Charlie’s Article entity:

public String Title
{
    get
    {
        return this.title;
    }
    set
    {
        if (this.title != value)
        {
            this.title = value;
            MarkDirty();
        }
    }
}
private String title = String.Empty;

For reasons of performance and user experience, Charlie will not commit a business object to the database unless that object has actually changed (in which case its data is considered “dirty”). By using reflection, we could alter the private title field, so the object would not be marked as dirty. But reflection comes with the penalty of performance and complexity. Without relection, our Data Access code must work through the public Title property, so the object will be marked as dirty. But, when the object has been freshly retrieved from the database, it is not dirty, so this would an erroneous dirty flag.

The current solution in Charlie is to have the EntityManager “reset” the dirty flag on a freshly-retrieved object.

entity = this.Mapper.Retrieve(entity, criteria);
entity.MarkAfterLoad();
return entity;

This approach is a little clumsy, but it works just fine. And after all, a solution that works is a working solution. Even though this works, I have decided to replace this solution with the IObjectHelper interface. This way, my hand-coded CRUD methods have a single point of working with Charlie’s business objects. Later, if I switch to using the WilsonORMapper, the business objects will not need to change at all. That seems to be a good compromise between the devil I know and the devil I don’t.

by Alister Jones | Next up: Charlie has Cool URLs - Part 1 →

Charlie’s Alarm Clock

2006-05-08T13:52:00.000-07:00

Readers, if you have not already subscribed to WilsonDotNet.com, go and do it now. For just $50, you get so much useful stuff. Even if you don’t care to look at the code, the components themselves are worth the price. But the real value comes from looking at the code of those components. You are nearly guaranteed to learn a few things that will save you many hours on your own projects. If you place any value on your own time, you will appreciate that joining Paul Wilson’s website is simply the smart thing to do—the subscription will pay for itself many times over.

Sure, the above is a sales pitch. But this weblog is an honest account of everything I have learned while designing Charlie, and the benefit of subscribing to Paul’s website is something I am learning over and over again.

I have just now added Paul’s KeepAlive code to Charlie. The WebApplication class has had the following static constructor introduced:

static WebApplication()
{
    Timer.Instance.Elapsed += new EventHandler(KeepAlive);
}

Both the Timer class on the left and the KeepAlive handler on the right are lifted straight out of Paul’s WebPortal project. Every fifteen minutes, the Timer elapses and Charlie requests one of its own pages. This page request is all that is needed to stop ASP.NET from unloading the application. So this code effectively introduces an alarm clock that will wake Charlie up before it has the chance to fall asleep.

Because the Charlie application is kept awake, this means that the Cache is not unloaded. The Cache too was enhanced with an idea or two from the Wilson WebPortal.

So in the last two days, I have added three separate ideas from the Wilson WebPortal. Because all of Paul’s components are provided with source code, I was able to open Paul’s code in one window, open Charlie’s code in another window, and virtually copy the code. The code was not literally copied, since I tweaked Paul’s code to work with Charlie’s design. But having a source-provided component to work from meant that it took me only about an hour to accelerate Charlie’s performance. It would have taken me a few hours to first come up with the same ideas, and then many more hours to research the ideas and write the code. That is my subscription paid for, right there. And that is not yet taking into account the WilsonORMapper, which I will talk about in my next weblog entry.

In these days of spam and affiliate sponsors and viral marketing and so on, it is very hard to make a strong recommendation without it coming across as insincere. Yet the simple truth is that I have no vested interest Paul’s websites, but Charlie has benefitted so much from my subscription to his website that I must give credit where it is due. I have benefitted from my subscription, Charlie has benefitted, and you will too.

Thank you, Paul Wilson.

by Alister Jones | Next up: The Two Devils of SQL →

The EntityCache Object

2006-05-07T22:52:00.000-07:00

When I first designed the Entity System for Charlie, I gave the EntityManager properties for IsCacheable, CacheDependency, CacheItemPriority, CacheSlidingExpiration, and so on. With these properties, each concete Manager object can specify how caching should be applied to any Entity or EntityCollection objects it returns. For example, the RoleManager can specify whether the Role objects it returns should be cached and, if so, how they should be cached.

To start with, I set the IsCacheable property to false for every Manager. By effectively disabling the cache, I could concentrate on the functionality of the business objects, leaving caching as a final optimisation step. As I explained in my earlier weblog entry, I brought forward the plan to implement caching of the business objects. I left all of the caching properties in the base EntityManager class, but moved the actual caching work to a new EntityCache class. That way, I can change the caching mechanism without affecting the EntityManager object, which doesn’t really care how caching is implemented.

Let’s take a quick look at how caching works in the context of the Entity System. Specifically, it is the job of the base EntityManager object to test whether the requested Entity already exists in cache. Here is a cut-down version of the relevant code:

public abstract class EntityManager
{
    protected Entity LoadEntity(EntityCriteria criteria)
    {
        String cacheKey = // to be discussed
        Object cached = this.GetEntityFromCache(cacheKey);
        
        if (cached != null && cached is Entity)
        {
            Object clone = ((Entity)cached).Clone();
            entity = (Entity)clone;
        }
        else
        {
            entity = this.Mapper.Retrieve(entity, criteria);
            this.AddEntityToCache(cacheKey, entity);
        }
        return entity;
    }
}

To see why the cached Entity is cloned before being returned to the calling class, please refer to my previous weblog entry.

The very first problem to be solved is what key value can we use to determine whether the cache contains the Entity being requested? It is not enough that we simply use the ID value of the Entity being requested. After all, an ArticleEntity with an ID value of 32 is not the same as a WeblogEntity with an ID value of 32. Further, it is not enough to use a key representing the entity type and the ID value (such as “Charlie.Articles.Article.32”), since localisation means that the cached entity may be the English version, when the incoming request is for the French version. This suggests that we could use a key that is a composite of the entity type, the culture, and the ID (such as “Charlie.Articles.Article.32.en-US”). The problem here is that the concrete ArticleManager may require more fine-grained caching, such as caching one version of the Article together with comments by visitors, and another version of the Article without comments by visitors. But the base EntityManager class cannot possibly know of all the caching variations required by the concrete manager classes. So, how can the base EntityManager class implement caching, when it cannot know of the variations required by the concrete manager classes?

Fortunately, this was a very easy problem to solve. We need only look closely at the LoadEntity method of the base EntityManager class to see that there is something that uniquely describes the Entity being requested, and that that something properly reflects all of the caching variations needed:

public abstract class EntityManager
{
    protected Entity LoadEntity(EntityCriteria criteria)
    {
        String cacheKey = criteria.ToString();
        // rest of the class
    }
}

To understand how the criteria reflects all of the caching variations needed, here is a pretend ForumCriteria class:

public class ForumCriteria
{
    public Int32   ID;
    public Boolean LoadById;
    
    public String  Name;
    public Boolean LoadByName;

    public Culture Culture;
    public Boolean LoadByCulture;

    public Boolean LoadReplies;

    public Boolean LoadAvatars;
    
    public override String ToString()
    {
        return
            "Charlie.Forums.Forum" +
            ID + LoadById +
            Name + LoadByName +
            Culture + LoadByCulture +
            LoadReplies +
            LoadAvatars;
    }
}

When the base EntityManager class calls criteria.ToString(), it will end up with a unique cache key, such as:

Charlie.Forums.Forum.32.True..False.en-US.True.True.False

This means that the base EntityManager will always cache entities in a way that precisely reflects how that entity was requested by the incoming criteria. Problem solved. We could actually use reflection instead of forcing the developer to implement a custom ToString() method. However, in the comments in my code, I have said the following:

//  Yeah yeah, we could use reflection to inspect this criteria class.
//  But, what's the point of using Cache for speed if we use slow old
//  reflection to get the cache key?

The second problem to be solved was getting the EntityCache class to return lazy-loaded entity objects, and not fully-loaded entity objects. My last two weblog entries described the problems I faced here, and the solution provided by cloning entities.

The final problem to be solved was whether to use the intrinsic ASP.NET cache, or implement a custom caching mechanism. Initially the plan was to use the intrinsic cache, which is why the EntityManager class provides the following properties (simplified as fields):

public abstract class EntityManager
{
    protected Boolean                  IsCacheable;
    protected CacheDependency          CacheDependency;
    protected CacheItemPriority        CacheItemPriority;
    protected CacheItemRemovedCallback CacheItemRemovedCallback;
    protected DateTime                 CacheAbsoluteExpiration;
    protected TimeSpan                 CacheSlidingExpiration;
}

This allows each concrete manager (such as the RoleManager) to precisely describe whether the entity handled by this manager (the Role entity) should be cached and, if so, how it should be cached. If you consider that the CacheDependency property can be a SqlCacheDependency, you will appreciate just how flexible is the intrinsic ASP.NET cache.

I then considered using a custom cache, based on that used by Paul Wilson in his WebPortal project. But I eventually decided against this approach. To start with, my recent foray into caching proved that I don’t have a great understanding of reference types, and I had also learned that I don’t have a solid understanding of static classes. More importantly, I felt that the flexibility of the intrinsic ASP.NET cache was too beneficial to give up. I decided that Charlie would use the built-in cache, but with two enhancements.

The first enhancement was to use the idea provided by the Wilson WebPortal, even though I would not use its implementation. Specifically, if the concrete manager specified that the CacheItemPriority was High (which is the highest setting available), then the EntityCache would not only add a clone to the built-in cache, it would also add the same clone to a custom HighPriorityCache. This custom HighPriorityCache was nothing more than a static class that exposed a Hashtable. Because the cached item was being referenced by a static class, that item would not be removed from the cache. Without that reference, the item could be removed from the cache. All this really means is that when the EntityCache sees that an entity has a CacheItemPriority of High, it performs a little trick that moves the priority from high to permanent—that item simply will not be expired from cache within the lifetime of the Charlie application.

The second enhancement was to come up with a way to immediately expire objects from the cache. With the intrinsic ASP.NET cache, it is not possible to forcibly expire an item from the cache. You cannot expire the item, nor can you set it to null. However, I desperately wanted to give Charlie the ability to instantly expire any cached Entity or EntityCollection. The reason I wanted to do this is because experience on other websites has shown that it is a pain in the ass when you make changes to the website, but those changes are not reflected for up to 15 minutes (or whenever the cache expires). I don’t want to subject Charlie’s website owners to that same frustrating experience.

The solution was very simple. The EntityCache class works only with the EntityManager class, which in turn works only with Entity and EntityCollection objects. So, I gave both the Entity and the EntityCollection classes an internal IsExpired property.

internal Boolean IsExpired
{
    get
    {
        return this.isExpired;
    }
    set
    {
        this.isExpired = value;
    }
}
private Boolean isExpired;

Then, when the EntityCache class receives a request to remove an item from its cache, it looks first to see whether that item is in cache. If that item is in cache, it cannot actually delete that item—the ASP.NET cache does not allow this. Instead, the EntityCache class marks the cached Entity or EntityCollection as being expired.

internal void Remove(String cacheKey)
{
    Object cached = HttpContext.Current.Cache[thisKey];
    if (cached != null)
    {
        if (cached is Entity)
        {
            ((Entity)cached).IsExpired = true;
        }
        else if (cached is EntityCollection)
        {
            ((EntityCollection)cached).IsExpired = true;
        }
    }
}

Then, when the EntityCache receives a request for a cached Entity or EntityCollection, it will look to see whether there is a cached item and, if so, whether that item is marked as expired. If there is no cache item, or if there is a cached item but it is marked as expired, the EntityCache will return a null value.

internal Object Get(String cacheKey)
{
    Object cached = HttpContext.Current.Cache[cacheKey];
    if (cached == null)
        return null;
    if (cached is Entity)
    {
        if (((Entity)cached).IsExpired)
        {
            return null;
        }
        else
        {
            return cached;
        }
    }
    else if (cached is EntityCollection)
    {
        if (((EntityCollection)cached).IsExpired)
        {
            return null;
        }
        else
        {
            return cached;
        }
    }
    return null;
}

When the EntityManager class receives a request to Save an Entity or EntityCollection, the manager will automatically expired any cached copies. When the Entity or EntityCollection is next requested, it will be drawn from the database, which means it will always reflect the most recently committed changes. This was the problem to be solved, and it was solved with minimal code changes. (By the way, I know that the above code could be compressed. However, I feel no motivation to compress code if it would make it harder to understand what is going on. A lot of my code will look relatively verbose, but that’s fine by me. It’s fine by the compiler, too.)

After a few bumbling steps, Charlie’s Entity System is now pretty darn fast. It now makes use of an enhanced cache that stops high priority items from expiring, and that allows for immediate expiration of updated items. And there is no O/R mapping or reflection to slow down the Entity System.

by Alister Jones | Next up: Charlie’s Alarm Clock →

A Word on Lazy Loading

2006-05-06T15:12:00.000-07:00

In my previous weblog entry, I described my fundamental misunderstanding of how the ASP.NET Cache works. But while I finally understood that the cache works like a shadow, I needed Charlie’s cache to work like a box. The reason I needed a box is that Charlie’s business objects use lazy loading. And what, you ask, is lazy loading?

One of the fundamental business objects in Charlie is the Document entity, which represents a document on the internet. The Document entity includes a Containers property, which exposes a ContainerCollection. For example, the Homepage document might include a Header container, a Menu container, a Content container, and a Footer container. Another object can “get” these containers through the Document.Containers property. So, a fully-loaded Document object would look like this:

What you will see from the above diagram is that the Document object contains to the Containers object. You will also notice that the Containers object is a much bigger object than the Document object itself. (If you took away the Containers object, the Document object would collapse to a much smaller size.) The Document object just includes simple properties such as its URL and its Title. The Containers however include Models, Views, Controllers, and much more.

If a particular Document object is being presented to the user, then we do want these Containers. The Containers will hold the header, content, footer, and everything else in the Document. So to display the Document, we need its Containers.

However, if we are presenting a site map page or similar, then we do not need these Containers. All we want is the Document with its URL and its Title. Loading up all of those Containers for every Document on the site map page will incur a major database hit, when we won’t be using the Containers anyway.

Lazy loading is a technique by which we load up just the main business object, and not all of its child business objects. In this example, we would load up the Document object, but leave the Containers property as null. Here is how the code looks, followed by an illustration of the half-loaded Document business object.

public class Document
{
    public ContainerCollection Containers
    {
        get
        {
            // We'll come back to this
        }
    }
    private ContainerCollection containers = null;
    
    // rest of class
}

You will see that, to start with, the private containers field is null. A given Document’s containers are not loaded when the Document is retrieved from the database. This means that any Document used is nice and light, and does not carry a heavy Containers collection. Again, if we are working with a site map or similar page, the Containers will not be used, so it is a waste to load them. However, if another class requests the Containers from a given Document, then that is the point at which the Document will have to go and fetch its containers from the database. Here is how that looks in code, and how you might envisage the process.

public class Document
{
    public ContainerCollection Containers
    {
        get
        {
            if (this.containers == null)
            {
                this.containers = 
                    ContainerManager.GetCollectionByDocumentId(this.Id);
            }
            return this.containers;
        }
    }
    private ContainerCollection containers = null;
    
    // rest of class
}

This process of deferred loading of child business objects is known as lazy loading. I have said that lazy loading means that Charlie wants the cache to work like a box, and not like a shadow. Why?

Let’s look at how a Document might be loaded. (This is not how it looks in Charlie. This is just a simple example.)

public class DocumentManager
{
    public Document GetDocumentById(Int32 id)
    {
        String cacheKey = "Charlie.Document." + id.ToString();
        
        // If we have the document in cache, return it
        Object cached = HttpContext.Current.Cache[cacheKey];
        if (cached != null && cached is Document)
        {
            return cached as Document;
        }
        
        // Get the document from the database
        Document document = DataAccessLayer.LoadDocumentById(id);
            
        // Store the document in cache
        HttpContext.Current.Cache.Insert(cacheKey, document);
        
        // Return the newly-loaded document
        return document;
    }
    
    // rest of class
}

To put the code into words, the DocumentManager first checks whether the requested Document has been loaded before and has been placed into the cache. If so, that cached version will be returned. If the requested Document has not been loaded recently, then it will be fetched from the database. That newly-fetched Document will be stored into cache, so that the next request for the same Document can be served from the cache rather than incurring another database hit.

Let us presume that the DocumentManager receives a request for a Document that has not been loaded recently. It requests the document from the Data Access layer. Keeping in mind that the Document uses lazy loading, it has a whopping hole where its Containers would be. Here then is a diagram of this newly-loaded Document.

The DocumentManager will store this new Document in the cache.

The original document is returned to the application. The application is presenting a single full page, and not presenting a site map or similar page. So, it will call the Document.Containers property in order to get the containers it needs to present the Document. Because the Containers property is lazy loaded, this will force the Containers to be fetched separately, and used to “fill out” the original Document. What is important to note is that because the cache works like a shadow, the cached version of the document is also filled out.

The document will be returned to Peter, who is a visitor to the website. Now, Samantha requests the same document. This time, the DocumentManager will see that it does have a cached version of the same Document, so it will return the Document from cache, rather than loading it from the database. However, because the cached version shadowed changes to the last-requested original version, then the cache will return the filled-out version of the Document, and not the half-empty, lazy-loaded version of the same Document.

This is no good. We don’t want the DocumentManager to sometimes return a lazy-loaded version, and sometimes return a fully-loaded version. Peter and Samantha may have different authorization roles, or may have different user preferences, so the Document needs slightly different Containers for each user. (Or Peter may be an administrator, and has changed the Containers or their content.) To allow Peter and Samantha to view slightly different versions of the same Document, we need the DocumentManager to always return a lazy-loaded Document, and never return a fully-loaded Document. Moreover, the class that requests a Document from the DocumentManager should never have to worry about the possibility that the returned Document may be an old, filled-out version. That calling class should be able to presume that the returned Document is always a new, lazy-loaded version. For this to be true, we need the cache to work like a box, and not like a shadow.

Before Charlie, I had never used lazy-loaded business objects. Before Charlie, I had never used localised business objects. Before Charlie, I had never used fine-grained security on business objects. So, when my inexperience with secured, personalized, localised, lazy-loaded business objects was combined with my misunderstanding of the cache, all hell broke loose in Charlie. The ultimate solution was to clone an object going into the cache, and then clone an object coming out of the cache.

public class DocumentManager
{
    public Document GetDocumentById(Int32 id)
    {
        String cacheKey = "Charlie.Document." + id.ToString();
        
        // If we have the document in cache, get it
        Object cached = HttpContext.Current.Cache[cacheKey];
        if (cached != null && cached is Document)
        {
            Document original = (Document)cached;

            // So that changes are not shadowed, 
            // return a clone, not the cached original.
            Document clone = original.Clone();
            return clone;
        }
        
        // Get the document from the database
        Document original = DataAccessLayer.LoadDocumentById(id);
        
        // Clone the original document
        Document clone = original.Clone();
            
        // Store the clone in cache
        HttpContext.Current.Cache.Insert(cacheKey, clone);
        
        // Return the newly-loaded, original document
        return original;
    }
    
    // rest of class
}

By cloning an object before it is placed in the cache, and then cloning that object as it comes out of the cache, the cache works like a box and not like a shadow. And by having the cache work like a box and not like a shadow, the integrity of lazy-loaded business objects is preserved.

One final note is that cloning a business object is not as easy as calling the Clone method, since by default there is no such method. The Clone method in Charlie’s business objects is a custom method of the underlying Entity System. This custom method is derived from Expert C# Business Objects.

Wow, what a long weblog entry. It is my hope that it saves someone else from experiencing the same problems that I experienced with caching lazy-loaded business objects.

by Alister Jones | Next up: The EntityCache Object →

The Cache is a Shadow, Not a Box

2006-05-03T16:15:00.000-07:00

Ahhh, the trials of being a self-taught developer. Just when you think you’re getting a handle on things, you receive fresh evidence that you don’t know jack. One week ago I saw Beginning Programming for Dummies in my city’s technical bookshop, and I allowed myself a wry smile, thinking, “I’m way beyond that.” Today, I plan to go into the city and buy that book. In the last week I have discovered that I really don’t understand the fundamentals of programming.

I have long known that application optimisation should be a final polishing step, not an initial design step. For that reason, my Entity System for Charlie included a few basic caching features, but I had disabled them. The plan was to enable the caching after the rest of Charlie had been developed, as part of a general optimisation phase. However, I brought forward the caching phase, as I wanted to make use of the custom cache code from Paul Wilson’s WebPortal project.

So I enabled the existing caching features of the Entity System and—snap!—the security stopped working. I could not figure out why. Worse, I sent myself off on a wild goose chase. My debugging efforts suggested that the problem was how my business objects were being populated. (If populated from the database, they worked. If populated from the cache, they did not work.) But I could not figure out where I had gone wrong.

Last night I was tossing and turning in my bed, worrying about Charlie. I’d like to say that a supermodel elbowed me in the side and told me to cut it out. But that had been the night before. Last night it suddenly occurred to me that perhaps I was misunderstanding the basics of the cache. Now this may seem so obvious that I should be ashamed of myself. However I had long been using the cache without problem, so it did not occur to me that I might have a gross misunderstanding of it. I mean, we stick an object into cache, and later get it out again. Who could get that wrong? Turns out, I was getting that wrong.

Let me tell you how I thought the cache worked. We’ll start by creating a simple ArrayList object. Here is the code, and a diagram illustrating how I viewed this object in my mind’s eye.

ArrayList original = new ArrayList();
original.Append("1");

Let’s put the object in cache. Again, here is the code, followed by a diagram illustrating how I envisaged the cache working.

Cache.Insert("original", original);

Clearly, I viewed the cache as being like a box into which we can place objects that we later want to retrieve. To show more clearly how I saw the box working, let’s add a few more items to the original object.

original.Append("2");
original.Append("3");

In my mind’s eye, I saw the cached object as being unaffected. Because that is how I saw the cache working, my belief was that if I retrieved the object from cache, it would be in the same state as it went into the cache.

ArrayList copy = Cache["original"] as ArrayList;

As I have suggested, I took it as a given that the cache worked like a box. When the object goes in, it remains unchanged until we take it out again. This seems so natural to me that it never occurred to me that the cache could work any other way. (But if I properly understood reference types, I might have realised my error. This is why I think I need to read Programming for Dummies after all.)

The belated insight I had last night was that the cache may not work like a box—that my basic assumption was wrong. I have just now created a test webpage in Charlie, and used its logging plugin to record what happens to an object placed in its cache. Here is what I discovered, starting again with the original ArrayList object.

ArrayList original = new ArrayList();
original.Append("1");

Again we put the original object in cache. This time, we envisage the cached object as being a shadow of the original object.

Cache.Insert("original", original);

If we see the cached copy as working like a shadow, and not like a box, we can properly predict what will happen if we then change the original object.

original.Append("2");
original.Append("3");

With this correct mental picture, we can also understand what happens when we retrieve the cached object.

ArrayList copy = Cache["original"] as ArrayList;

Because I had the wrong mental picture of the cache, I was using it improperly in the Entity System for Charlie. The subsequent problem to be solved is that the Entity System needs the cache to work like a box, not like a shadow. Fortunately, this is easy. If you don’t want the cached object to shadow the changes to the original object, you need to cache a clone of the original object. The clone will be unaffected by any changes to the original object.

ArrayList clone = original.Clone() as ArrayList;
Cache.Insert("clone", clone);

I cannot believe that I am the only person to have mistakenly viewed the cache as working like a box. If so, I hope this weblog entry may help others.

by Alister Jones | Next up: A Word on Lazy Loading →

Sunken Ships and Pirates

2006-04-19T17:29:00.000-07:00

I have finished rebuilding the main parts of the Wilson WebPortal. I have been so impressed with its simplicity and its effectiveness that I have given great thought to whether to move the best of Charlie to the WebPortal, or the best of the WebPortal to Charlie.

While I would naturally have some misgivings about kicking the chair out from under Charlie, I am very cognisant of the economic concept of a sunk cost. Specifically, the time and effort I have put into Charlie is a sunk cost—a cost incurred in the past that should have no bearing on future decisions. All that is relevant to future decisions is future costs. In this case, the future cost will be either the time taken to move Charlie to the WebPortal, or the time taken to move the WebPortal’s ideas to Charlie. Whichever approach provides the greatest benefits to costs ratio is the approach to take, regardless of the sunk cost of Charlie’s past development.

So what would be the benefits to switching to the Wilson WebPortal? I can see three benefits. First, I would be using a website framework that would be developed and maintained by someone else, leaving me free to focus on creating websites rather than the underlying framework. Second, I could use any modules created by Paul or other developers. Third, I could potentially sell any really good modules that I might create.

What then would be the costs involved in switching to the Wilson WebPortal? I can see two costs. First, I would need to spend time learning the new framework. Second, I would be limited to the functionality that the framework provides. Most notably, the current version of the Wilson WebPortal does not support globalization in its code or in its database schema, so I would be giving this up.

Now, since I am a subscriber to Paul’s website, I have the source to the Wilson WebPortal. Strictly speaking I could tweak the framework’s code to do whatever I like. However, I cannot see that this is a sustainable option. The moment I touch the core framework, I will be taking it down another development path that will prevent me from then “upgrading” to any future versions released by Paul. And if I cannot upgrade to future versions, then that immediately kills the benefit of having someone else maintain the base framework.

To solve this problem, I considered introducing a façade layer. Any “tweaks” could be applied to this façade layer, so that the underlying WebPortal framework could be kept in sync with any releases from Paul. However, while this sounds like a workable approach, a façade layer can only introduce relatively superficial enhancements to the underlying framework. Globalization, for example, is too deep an enhancement to be implemented in a façade layer. Globalization affects everything from threading to the business objects to the database schema to the persistence code. Even more problematic is that such an enhancement to the framework would be a breaking change. Even if I submitted my code revisions to Paul, he could not include them in future versions of the WebPortal without breaking everyone’s existing website. These concerns mean that I cannot see that tweaking the WebPortal framework is a sustainable approach. In turn this means that by moving to the Wilson WebPortal, I would be giving up Charlie’s working globalisation, which would be a major cost to incur.

So what would be the benefits of taking the approach of a pirate? I could take my cutlass to the Wilson WebPortal, steal its treasures, and bury them in the sand of Charlie under the light of the full moon. (This would be within the terms of use of the WebPortal, by the way.) I can see two benefits. First, I can apply some of Paul’s great ideas to Charlie, most notably his KeepAlive code and custom Cache code. This would mean adopting about 100 lines of Paul’s code. It’s not much, but it’s code that I could not have written myself. Second, Charlie would stay a bespoke application, whereas the WebPortal is a general application, and having a bespoke application has many attendant benefits.

What then is the cost involved in sticking with Charlie over moving to the Wilson WebPortal? Well, the cost is the same cost that has long been bothering me: that I’m creating and maintaining everything myself.

So which approach provides the greatest ratio of benefits to costs? Should I sink the Charlie ship and move to the WebPortal ship? Or should I keep the Charlie ship afloat and plunder the treasures of the WebPortal ship?

After thinking about this at some great length, I have decided to keep the Charlie ship afloat, and let it take the pirate approach of plundering goodies from any other ship it finds on the high seas of ASP.NET. Unlike pirates, however, I’ll only take what I’m entitled to take.

by Alister Jones | Next up: The Cache is a Shadow, Not a Box →

The Provider Model, Unveiled

2006-04-18T17:33:00.000-07:00

In my previous post, I suggested that only a small percentage of ASP.NET developers could describe where the new provider model fits within a properly-architected application. I suggested that the reason so few developers understand the technology they are using is because .NET envangelists are typically magicians who perform tricks, rather than teachers who explain concepts.

I am still learning ASP.NET version 2.0, but I’ll nonetheless tell you my current understanding of the provider model. Let’s start with a look at the common layers in a properly-designed application.

The magicians will tell you that the default providers work against SQL Server or SQL Server Express, and if you want to use another database you must implement a different provider. You can then place on your .aspx page a control such as the Login control, and point that control at your custom provider. What this suggests is that the Login control is in the Interface layer and the provider is in the Data Access layer.

Even the naming of the providers suggest that they belong in the Data Access layer. The default Profile provider is the SqlProfileProvider. As one alternative, Microsoft provides a downloadable AccessProfileProvider.

If you read the magicians’ articles and blog entries, then visualising the Providers as belonging to the Data Access layer is about the only possible conclusion. But anyone with a properly-architected application will shake his or her head at the magician, and grumble about Microsoft eschewing proper architecture in favour of the slap-it-in-place-in-Visual-Studio approach. Anyone with a properly-architected application will have a User business object and probably a Role business object, both of which the above design will avoid much like a pedestrian will side-step dog turd.

The above illustrates one of the reasons I was so concerned about ASP.NET version 2.0. How can it be a good thing to use these database providers that skip over business objects?

What has finally occurred to me however is that the providers do not belong in the Data Access layer, despite the names of the providers, and despite what the magicians tell me. No, the providers belong in the Controller layer. In the case of the default SQL provider, it acts like the other layers are not even there. The default providers work as though the other layers have been erased, and it’s okay to talk directly to the database.

For small applications, it probably is okay that the providers talk directly to the database. But for a properly-architected application, let’s look at what it means that the providers are in the Controller layer. The providers “hide” the business objects from the interface layer. All the interface layer sees is the provider, and has no idea whether that provider is talking directly to a SQL database, talking directly to an Oracle database, talking to business objects, talking to web services, or talking to Felix the Cat.

In my look at Paul Wilson’s practical approach to business objects, I said that he uses a class-in-charge design where the user is represented by one User class, and the authorization role is represented by one Role class. In his WebPortal project, he uses a custom Profile provider that talks to these business objects, rather than talk directly to the database. Even better, the project’s use of an O/R Mapper means that the same provider and same business objects could use a SQL database, an Oracle database, or one of many other databases. So here is the location of Paul’s custom provider in the context of his WebPortal project. Notice that the provider is talking to the business objects, and not to the database.

Charlie does not use the same class-in-charge design used by the Wilson WebPortal. Rather, it uses a five-part Entity System. I can introduce a provider into Charlie’s Controller layer, which talks directly to the relevant Manager classes, in which case the architecture would look as follows.

What is great about the provider model—once you ignore the magicians—is that it allows the same rich server controls to work with any architecture. The Login control will work with a shallow architecture where the provider works directly with the database, or with a multi-layered architecture where the provider works with business objects (such as in the Wilson WebPortal), or with a multi-layered architecture where the provider works with manager objects (as in Charlie).

In many respects, the providers are the controllers in the popular model-view-controller pattern. The server control provides the view, the business objects provide the model, and the provider provides the controller. The providers are not true controllers, since they are not designed according to the true MVC pattern. However, they do provide the same separation of responsibilities that is the reason why the MVC pattern is so popular.

We can see then that the provider truly enhances the architectural integrity of .NET applications, and this is a very good thing. My earlier concern about providers was wholly unfounded, because I did not understand where the providers exist in the context of a layered application. I don’t think my misunderstanding is the result of me being a fool. (I may be a fool, but I don’t think it caused this misunderstanding.) Rather, I really do think .NET evanglists are magicians who focus on details (the tricks), rather than teachers who look at the bigger picture.

Now that I understand the provider model, I can see how I can maintain Charlie’s existing architecture while staying true to the design of ASP.NET version 2.0. Even better, this fleshes out Charlie’s architecture even further, since previously the Controller layer was something of a desolate landscape—there was nothing to see.

Finally, a disclaimer of sorts. This is my current understanding of the provider model, and it is an understanding brought about by my look at the Wilson WebPortal. If my understanding is wrong, or if I have misrepresented the Wilson WebPortal in my diagram above, the error is wholly mine. So if I am right, the thanks go to Paul. If I am wrong, the blame goes to me.

by Alister Jones | Next up: Sunken Ships and Pirates →

The Magician and the Teacher

2006-04-17T17:41:00.000-07:00

Tom has come home from his first high-school physics class, and already he is sensing that he’s going to fail physics. All those numbers and formulas. He goes to his Dad and shows him the formula for momentum, p=mv, and asks his Dad what it means. “That’s simple, son; it’s just mass times velocity.” He grabs his calculator and says, “If the mass is four and the velocity is three, then the momentum is,” wherein he pauses to let Tom type the equation into the calculator, “twelve. Simple!” He pats his son on his head, and walks out to watch some TV. Along the way he thinks, I hope Tom is good at football, because his maths sure sucks. And Tom is thinking, I hope I can grow up to be a football player, because I’m going to fail physics.

Alison is in the same class as Tom, and asks her mother the same question. Her mother takes her by the hand and together they go to the spare room. Her mother pulls out the bowling ball that her husband bought her, hoping that she’d show an interest in his one hobby. She then finds a baseball and places it beside the bowling ball. Feeling slightly guilty, the mother says, “Imagine that you’re standing over there,” as she points to the doorway, “and each of these balls is going to roll toward you and hit your feet.” Alison looks a little amused, so her mother presses on with the analogy. “Imagine that they are both rolling toward you at about the speed you can run. Which one is going to hurt more?” Alison thinks the question is silly, but says, “The bowling ball is going to hurt, because it’s so heavy.” To which her mother says, “Correct.” Her mother then goes on to say, “Now, imagine that your baby brother has pushed the bowling ball toward you, but a cannon has shot the baseball toward you. Which one will hurt more?” Alison says, “The baseball, since it’s going so fast.” With the demonstration complete, Alison’s mother takes her back to her textbook where they look at the formula for momentum, which is p=mv. The mother explains that m is the weight of the bowling ball or the baseball, and v is the speed at which the ball is going. Harking back to the demonstration, her mother explains that sometimes a baseball will carry more momentum, and sometimes a bowling ball will carry more momentum. They then do the same sums that Tom did.

Guess who passed the first physics test? Well, actually, Tom and Alison both failed, since they had fallen in love and spent the entire classes passing love notes back and forth. But had love not blinded them both, Alison would have passed because her mother took the time to teach her, while Tom’s father only took the time to show him.

What this little story shows is that it takes me three paragraphs to make a single point. And the point is that a formula and a calculator is like a magic trick and the magician—they hide from you the reality. If you know the formula and can work the calculator, then all you know is the formula and the calculator. You know the trick but not the reality. To see the reality, you need a teacher, not a magician.

Let’s look at another equation that has become common in the ASP.NET world:

control + provider = functionality

All over the internet, and in books about ASP.NET 2.0, we have magicians repeating the formula and telling us what buttons to press on our calculator (Visual Studio). But Tom will not learn physics this way, and we will not learn ASP.NET this way. All we learn is the trick. Where oh where are the teachers who will explain the reality behind the trick?

I have become increasingly frustrated with the magical approach taken by .NET evangelists. Put one control and one provider into a top hat, wave a wand over the hat, and—abracadabra—a bunny appears. That’s a trick. Where are the teachers who will explain the reality behind the trick?

To bring this rambling weblog into context, let’s look at the architecture for Charlie. If I want to align Charlie with the direction that ASP.NET is taking, where in that architecture would a provider fit? Now Charlie’s architecture is so familiar that any architectural question that troubles Charlie must trouble thousands of other applications too. So for those of us who attempt to create properly-architected applications, where does the new .NET provider model fit within this well-known n-layered design?

How many users of ASP.NET version 2.0 could answer this question of where the provider model fits within a properly-architected application? One in two? One in five? One in ten? I would hazard a guess that it’s a very low percentage indeed. Why? Because .NET is being evangelised by magicians, and not by teachers.

This is one of the reasons why I was initially fearful of ASP.NET version 2.0. All the evangelists were magicians pulling rabbits from their hats and scarves from their mouths. Impressive, maybe, but pretty darn useless in the real world. I was fearful that ASP.NET was nothing but a bag of tricks that had no place in a properly-architected application. This is a real shame, because there is a very sweet logic to ASP.NET version 2.0 that is completely at home in an n-layered application design. Rob Howard and Scott Guthrie and the team for ASP.NET version 2.0 have done a sterling job, but the evangelical magicians are doing the ASP.NET Team a complete disservice. I truly feel that Microsoft should seek evanglists who are teachers, not magicians. Teachers can explain what Microsoft has done, why it was done, and how customers can use what it’s done. Leave the magicians to entertain Tom and Alison and other kids that their birthday parties.

by Alister Jones | Next up: The Provider Model, Unveiled →

The Practical Man

2006-04-16T17:45:00.000-07:00

A few moons ago, I was unsure of how to start Charlie. Fortunately, by working my way through Cuyahoga, I gained the inspiration I needed to move forward. As this new moon shows its face, I need a guide to what ASP.NET version 2.0 can contribute to Charlie. For this reason I am working through the Wilson WebPortal in precisely the same way that I worked through Cuyahoga: by rebuilding it from scratch. Once I have learned the best bits of ASP.NET version 2.0 and the best bits of the Wilson WebPortal, I will determine how I can combine those strengths with the strengths of Charlie. In doing so, I hope to simultaneously overcome the major weakness of Charlie: that I’m creating everything from scratch.

The Wilson WebPortal is free to use, but its source is available only to those who subscribe to WilsonDotNet.com. I cannot then show any code on this weblog, but I’d definitely like to talk about some of the lessons I am learning by working through its code.

The first lesson is actually about Paul himself: he is so darn practical. Way back when ASP.NET was new, many developers complained about the time needed to serve the first webpage request after a period of website inactivity. ASP.NET was designed to unload applications that had not been active for a while. The problem was that websites without constant traffic would unload, and the next visitor would experience a slow-loading first page. Rather than complain about the problem, Paul took the more practical approach of solving the problem. He introduced what amounted to an alarm clock for an ASP.NET application, that would keep waking the app up before it had the chance to fall back to sleep. Naturally, the Wilson WebPortal includes this alarm clock.

As I started working through the Wilson WebPortal, I came across the following comments introducing a custom Cache class:

// I use my own cache instead of HttpRuntime.Cache to better control lifetime.
// My experience on shared hosts is that HttpRuntime.Cache drops far too often.
// So this is intended to be a major performance win, although it may look odd.

This is so practical. How many of us would simply use HttpRuntime.Cache without giving it a second thought? And how many of us would then curse about our website still being slow even though we were using the built-in cache? With its whitespace removed, the code for Paul’s custom cache is about 50 lines. In my opinion, it is this sort of practicality and simplicity that distinguishes Paul from most other developers. He’s a practical man.

Another area where Paul puts practicality first is in the design of his business objects. Every single business object in Charlie is based on the five-part Entity System. The benefit is the simplicity of each class taking on a single responsibility. The penality is the complexity involved in maintaining a large number of classes and the means by which those classes communicate. Paul takes precisely the opposite approach. A business object is represented by a single class. The benefit is the simplicity of having one “class in charge”—everything to do with the user is in the User class, everything to do with the portal is in the Portal class, and so on. The penality is that these single classes are relatively complex, as they take on numerous responsibilities. I asked Paul about this on his own website, so you can read Paul’s own rationale: About the responsibilities of the classes in Wilson.WebPortal.Core. The architects can argue about the merits of a class-in-charge design, but what cannot be denied is the resulting simplicity and practicality that the design achieves. Paul also refers to the utility of this design in the context of ASP.NET 2.0 providers. I haven’t yet come to terms with providers, so I cannot yet see the connection—but I’ll get there.

Right now, Charlie is not too far behind the capabilities of the Wilson WebPortal. But what the Wilson WebPortal does, that Charlie does not, is stay true to the design of ASP.NET version 2.0. I feel sure that this is a major strength of the Wilson WebPortal, and a major weakness of Charlie. I haven’t yet decided what to do to address this weakness (if anything), because I’m still working my way through the Wilson WebPortal as a step towards learning the best bits of ASP.NET 2.0.

by Alister Jones | Next up: The Magician and the Teacher →

Charlie. The Little Engine that Could

2006-04-15T17:47:00.000-07:00

In a much earlier weblog entry, I described how my stubborn nature has had an impact on the way in which Charlie has developed. Specifically, I have been tackling one problem at a time, and stubbornly refusing to solve the problem “the Microsoft way” if I think that way is either complex or flawed.

Another aspect to my person that is having an impact on Charlie is my lack of confidence in myself. I am acutely aware that I am not an experienced developer, and I am acutely aware that I’m just not that smart. For both of these reasons, I have been losing faith in what I’ve been doing with Charlie. I look at the work by the developers of Castle and think, “Now those guys are really smart, I should be using Castle.” I then look at the work by Paul Wilson and think, “Now that guy is really smart, I should be using the Wilson WebPortal.” I then look at the work by Telligent and think, “Now those guys are really smart, I should be using Community Server.” Quite simply, all of those coconuts are so smart, and I lack so much confidence, that I feel almost compelled to give up on Charlie. How could I create anything worthwhile unless I create it on top of their own projects?

While I spent time researching Castle, I kept hearing the word that means so much to me: simplicity. The documentation for Castle talks about how its components achieve simplicity, flexibility, re-use, and all the other good things in life that do not involve sugar or pheromones. What eventually occurred to me however is that they are using monstrously complex ways of achieving simplicity. On the other hand, I have been using simple ways of achieving simplicity. My lack of confidence tells me that if I’m not doing things the same way as these smart guys, I must be doing it wrong.

I have been looking through the code for Paul’s Wilson WebPortal. Without doubt, he has created a very clever system that makes good use of the best that ASP.NET 2.0 has to offer. As I study his code more closely, I can see that he is solving the same problems I have solved, but just in a different way. My lack of confidence tells me that if I’m not doing things the same way as this smart guy, I must be doing it wrong.

However, it has occurred to me that just because these guys are smart does not mean that what they do is right. And just because I am not as smart does not meant that what I do is wrong.

I have just fired up the current version of Charlie, and surfed between the pages of my sample website. You know what? While it is not as complex as Castle, or as clever as the Wilson WebPortal, it is working precisely how I want it to work. All of the pages are secure and localized, which is precisely the goal I had for Charlie at this stage in its development. Charlie is chugging along, getting the job done, and staying on track. Charlie is the little engine that could, and maybe I should have a little more faith in it. And in myself.

In my last web entry on the monkey or the gorilla, I commented on the difference between a framework application and a bespoke application. As I look now at the code for Charlie, I can see that much of its simplicity and effectiveness comes from the fact that Charlie has been designed as a bespoke application. I have been finding the simplest and most effective solution to each problem, without having to worry about whether other developers could work with the solution. In this regard too, Charlie is chugging along, getting things done in its own way. The little engine that could.

Now that I recognise that Charlie is properly a bespoke application, it answers the question of whether to move to Mono 1.0 or move to .NET 2.0. At this point in time, .NET 2.0 offers greater opportunities for me to find simple solutions to each problem, most notably with generics. I can, and should, make use of anything that enhances simplicity. By the time I’ve finished Charlie, Mono will probably have become compatible with .NET 2.0, so it’s really a moot question. I should not have worried myself about it.

So, after having been tempted away by other projects, I have come back to Charlie with a newfound belief in what I’ve been doing with it. Charlie is the little engine that could.

by Alister Jones | Next up: The Practical Man →

The Monkey or the Gorilla?

2006-04-14T17:49:00.000-07:00

The monkey is a reference to Mono, which is the spanish word for monkey. The gorilla is a reference to .NET, released by the 800-pound Microsoft gorilla. This weblog entry is an extension of my earlier entry concerning my indecision about which technology to use. Mono 1.0 is fully compatible with .NET 1.1, with the exception of Enterprise Services. Mono is not yet compatible with .NET 2.0, so there is a decision to be made about which technology to choose.

Right now, Charlie does not use a single capability provided by .NET 2.0. So, I could move it over to Mono 1.0 with very little effort. Or, I could evolve it to use .NET 2.0 with very little effort. But those are divergent paths that will not converge again until some time in the future. So, which path should I choose? Should I go with the monkey, or go with the gorilla?

I said in my earlier post that going with the Mono monkey would lessen my business opportunities. In a reply to that post, Brendan Ingram challenged that statement, and rightly so. I did not really explain what I meant when I said that. Brendan also concluded by saying that going with Mono may in fact increase my business opportunities. Whether that is true or not depends on the business model for Charlie. That is something to which I have not given due thought, which is a horrific oversight on my part. So, let me explain my thoughts on the matter.

First though, I want to put my thoughts in context. Specially, I want to repeat that I come into this industry from outside. As a result, the only sample applications that I have ever seen are framework applications, never bespoke applications. A framework application is like DotNetNuke, where its target is developers. They take the framework, plug in new pieces, flip a few preference switches, add some content, and away they go. A bespoke application is like Flickr, where its target is end users. Charlie started out as a bespoke application, as I had explicitly stated that its target users were my own clients, not other developers and their respective clients. But, because all of the sample applications that I have seen are framework applications, I keep losing sight of the fact that Charlie started out as a bespoke application. For example, I added plugins to Charlie in a way that would support other developers adding plugins too, even though that was not a goal for Charlie—it was just the way I have seen things done, so it was the way I did it too.

Now, framework applications and bespoke applications are naturally different, and require different decisions to be made to the same problems. Take for example the need to style a web application. If Charlie remains a bespoke application, then I can implement styling any damn way I please. No-one else will ever see the implementation, and no-one else will even care. But if Charlie were to become a framework application, then I need to implement styling in a developer-friendly way, which, in the context of ASP.NET 2.0, means using Themes and Skins. As a further example, take the possibility of adding AJAX functionality. If Charlie remains a bespoke application, I can implement client-side functionality any damn way I please. But if Charlie were to become a framework application, then I would need to implement AJAX in a developer-friendly way, which, in the context of ASP.NET 2.0, means Atlas.

I feel that I have two choices with Charlie. Leave it as a bespoke application, in which case I can go with either Mono or .NET. Or, I can make it a framework application, in which case I really need to go with the gorilla. Which should I choose? Well, that should be a business decision, and I see four ways in which Charlie can earn money.

The first way in which Charlie can earn money is the obvious way: find website clients and use Charlie to build the website. Charlie will be technology by which I can anwser “Yes” to questions such as “Can we edit the content of the website?” and “Can we extend our website later?” Whether Charlie is a framework application or a bespoke application makes no great difference here. But Mono would put a smile on the face of those website owners whose technical folk inevitably ask, “Do you use open-source software?”

The second way in which Charlie can earn money is by selling it, or its components, to other developers. Both the Wilson WebPortal and Community Server have a business model that includes this aspect. Going with .NET over Mono would help here, since the market is larger. More important, most Mono developers will be advanced developers who would have no interest in the simple Charlie. Conversely, many .NET developers are novice and intermediate developers who may certainly have an interest. But, this approach would mean that Charlie should become a framework application, not a bespoke application.

The third way in which Charlie can earn money is by releasing it as open-source software, and trying to encourage community development. Community development would mean that Charlie becomes a richer application than I could ever achieve by myself. Having a richer application would mean that Charlie could help me to obtain clients that I could not have obtained if I’d kept Charlie as a closed-source application. Going with .NET over Mono would help here too, simply because of the size of the market. However, I think the ASP.NET community is already saturated with open-source website frameworks, so I don’t think this is a realistic model for Charlie. Still, it is an option that needs to be considered.

The final way that Charlie can earn money is by providing an avenue by which I can earn incidental income. It has been suggested a number of times that I should write articles. If I were to move Charlie over to Mono, there would be very little that I could write about that would be of interest to others. MonoRail, for example, is just too hard for the average developer to install and use, and just too complex to write an article about (the example application on CodeProject doesn’t even work). But if I were to learn ASP.NET 2.0, I could write articles on the things that I learn along the way.

Does this rambling weblog entry explain why I sense that there are more opporunities if I were to follow the ASP.NET 2.0 path rather than the Mono 1.0 path (remembering that the paths are currently diverged, and may not converge again for a good long while)?

Another thing that concerns me (I worry about a lot, don’t I?) is that what I like most about Mono and its projects seem at odds with .NET 2.0. For example, the provider model of .NET 2.0 does not seem to be compatible with the O/R Mapping model that is prevalent in the Mono world. For another example, the rapid application development model of .NET 2.0 does not seem to be compatibile with the Inversion of Control model that Castle promotes. The goals of .NET 2.0 developers and Mono developers do not seem very compatible, which is why I sense that I need to make a decision on which way to go. I may have this completely wrong, and my concerns may be unfounded. However this weblog is a record of my ongoing learning and my increasing understanding, and this weblog entry notes my current thoughts on the monkey and the gorilla.

by Alister Jones | Next up: Charlie. The Little Engine that Could →

Evaluating the Wilson Web Portal

2006-04-13T17:50:00.000-07:00

In a previous weblog entry, I summarily dismissed the Wilson WebPortal because it takes the ASP.NET 2.0 approach to web applications, about which I have grave doubts. However, three separate concerns have made me reconsider the Wilson WebPortal and, therefore, the ASP.NET 2.0 approach.

The first concern is that, as I have been creating Charlie, I have been getting annoyed by the amount of repetitive code I have been writing. One area is that old chestnut: data access. I’m brand new to databases, and have written about thirty CRUD methods, but already I am super-bored with data access. The Wilson WebPortal naturally uses the WilsonORMapper, so I am super-tempted to take the same approach.

Another area where I find myself repeating the same darn code over and over again is the code to compare one business object to another. Every single one of my business objects has the following boilerplate code:

public new static Boolean Equals(Object object1, Object object2)
{
    if (object1 is Article && object2 is Article)
    {
        return ((Article)object1).Equals((Article)object2);
    }
    else
    {
        return false;
    }
}

public override Boolean Equals(Object object1)
{
    if (object1 is Article)
    {
        return Equals((Article)object1);
    }
    else
    {
        return false;
    }
}

public Boolean Equals(Article article)
{
    if (article.Id == this.Id)
    {
        return true;
    }
    else
    {
        return false;
    }
}

public override Int32 GetHashCode()
{
    return base.GetHashCode();
}

Because these tests must be against the final type (such as an Article, or Blog, or Webpage object), this code to test whether one business object is equal to another cannot be moved into a base class. (At least, not in .NET 1.1.) But a peek at the code for the WilsonWebPortal suggests that generics provides a way to avoid this repetitive code. To date I have been unable to find an article on generics that makes the slightest sense to me. Fortunately, if you look at the documentation long enough, you get an “ah-ha!” moment when generics actually starts making sense.

So, the problem of repetitive code, and the simple and direct way in which Paul solves this problem, has given me the first reason to look at the Wilson WebPortal.

The second concern that has been haunting me ever since I started Charlie is the knowledge that I am reinventing the wheel. There are already mature applications that do the same thing, but I feel that they get it wrong—I honestly believe that I can do better. The Wilson WebPortal is not yet a mature application. It is brand-new, written from the ground up using the best that ASP.NET 2.0 has to offer. It is therefore free of legacy crap (like the remanants of IBuySpy that shackle DotNetNuke) and free of accumulated crap (like all the gee-whizz junk that bloats DotNetNuke). It is a clean slate provided by a guy who does things in a way that I both respect and admire. Paul’s only weakness is visual design, but that is where I can add real value to what he has achieved with his new product.

The third concern that has been troubling me is the knowledge that I am actively avoiding ASP.NET version 2.0. While I am compiling against .NET version 2.0, I have not used a single aspect of this new technology. In everything I have been doing, I have been doing it in a non-ASP.NET 2.0 way. What this means is that I am driving my project off the sealed highway and onto a dirt track that leads somewhere else. I fear that I am taking an ideological approach rather than a practical approach, which will come back to haunt me when I find myself stranded with an application that is too different in too many ways.

For these reasons, I am going to work through Wilson WebPortal, and learn how it does its thing. I will then either move Charlie onto this project, or I will keep Charlie as a separate project and apply the lessons learned from the Wilson WebPortal. Either way, I will try to bring Charlie back onto the sealed highway that is ASP.NET version 2.0.

by Alister Jones | Next up: The Monkey or the Gorilla? →

“And if you don’t know where you’re going…

2006-04-12T17:55:00.000-07:00

…Any road will take you there” — George Harrison

When we last left off, I was about to embark on an investigation into the Windows Vista user experience. Because I was unable to get the preview to install, I have had to work from Microsoft’s documentation and screenshots. As I have been working through it, I have been horrified. With each operating system release, Microsoft describes it as a “bet the company” release. If that’s true then, on face value, Microsoft is in danger of losing that bet with Vista.

To market any product, the vendor needs to find and promote a unique selling proposition. Apple makes its USP clear: “Let the Mac be the centre of your digital lifestyle.” If my sister is anything to go by, Apple is delivering on its promise. My sis is in the process of getting rid of her stereo and her TV because her Apple equipment has made them unnecessary. She uses an iMac, iBook, and iPod, and she can get stuff done that I could not hope to do on my Windows XP machine.

What is the unique selling proposition for Windows Vista? “Bring clarity to your world.” What the hell does that mean? This lack of focus seems to be reflected in the product itself. There seems to be no rhyme or rhythm, logic or art, to any part of the Vista user experience. My concern about Windows Vista follows my concern about ASP.NET version 2.0.

I am starting to worry that Windows XP and .NET version 1.1 represent the pinnacle of Microsoft, and Windows Vista and .NET version 2.0 represent the first fall on the slippery slide to obsolescence.

Because of my concern, I have been contemplating switching to Mono. Projects like Castle and Cuyahoga demonstrate the true thought and sheer intelligence that Mono developers bring to the platform. What do we get over here on the .NET side? Crappy, useless stuff like the provider pattern (which is not even a pattern anyhow). Over on Mono they have complete implementations for Model View Controller, for Inversion of Control, for Aspect Oriented Programming, and for so much more. I know we can use these components in a .NET application, but I have a concern with stradding Mono 1.0 and .NET 2.0 that is beyond the scope of a weblog entry.

There are only two reasons why I have not yet jumped ship. (I’ve downloaded Mono and starting coming to grips with Castle and its goals, so I’m ready to jump.)

The first thing that holds me back from Mono is the Wilson WebPortal. The initial release is very awkward to use, but it shows great potential. Most importantly, for me, is that Paul Wilson never ceases to find simple and direct solutions. If ASP.NET version 2.0 is good enough for this clever cucumber, then it should be good enough for me, too. I’ll come back to the WilsonWebPortal in my next weblog entry.

The second thing that holds me back is that while it makes ideological sense for me to switch to Mono, I don’t think it makes much business sense to switch. If I stay with .NET, I can earn money from writing articles and selling components; if I switch to Mono, the opportunities are limited. If I stay with .NET, I can benefit from the truly awesome support provided by Microsoft and its developer community; if I switch to Mono, the support is limited. If I stay with .NET, I can use a greater number of free and commercial components; if I switch to Mono, the components are limited.

By the way, if you are not already aware, this is a flow of consciousness—I am resolving my indecision as I write this. My heart wants me to switch to Mono. My head tells me to stay with .NET. I started writing this blog entry immediately upon returning from a ride on my motorbike, where I was thinking through the arguments for and against moving to Mono, and whether I might make use of the Wilson WebPortal. My non-committal thought at the end of the ride was that I should stick with .NET rather than switch to Mono, and this weblog entry has sealed the deal. In the next weblog entry, I’ll address the question of whether I should make use of the Wilson WebPortal.

by Alister Jones | Next up: Evaluating the Wilson Web Portal →

Charlie Seeks Friend, Open to Romance

2006-03-31T17:57:00.000-08:00

I could not get the preview of Window Vista to install. The documentation on MSDN is broken—its popup JavaScript does not work—preventing me from viewing screenshots of the new user experience. But I do wish to view full-size screenshots from the new operating system.

Charlie is seeking a friend. Smokers and drinkers are okay. Kids are okay. Availability to a Vista installation is a must. If this is you, Charlie would love to hear from you. We’ll swap photos and see what the future brings. Open to romance.

by Alister Jones | Next up: And if you don’t know where you’re going… →

I’ll Take What’s Behind Door Number Three

2006-03-31T05:59:00.000-08:00

I have been in search of an interface convention for my application, Charlie. First I considered a conversational interface convention, then a PowerPoint interface convention, and then an explorer interface convention. My original plan was to evaluate many, many popular web applications such as Flickr and WordPress and the like. I have trialed both applications, but their usability pales against that provided by CityDesk. CityDesk gets it right because it builds on the Windows user interface, which also gets it right. This has convinced me that the third option, an explorer interface convention, is the right option for Charlie. I’m not going to trial more applications or come up with alternative interface conventions.

I have said before that my brain dines on Corn Flakes, so my challenge now is to break the Windows Explorer interface into its constituent parts. I started by skimming through the Apple Human Interface Guidelines, and immediately felt despair. Mac OS 9 was a fantastic operating system, and I miss it dearly. In the transition to Mac OS X, Apple seems to have put form over function.

I then attempted to read Microsoft’s Official Guidelines for User Interface Developers and Designers. What is ironic is that the documentation’s own interface is fucking awful! How about a “Next Page” link so I can go to the next page? In order to move to the next page, you have to find the current page in the treeview to the left, and click on the next leaf down. Worse, clicking on the link clears the screen and then loads up a whole new frameset, with a whole new copy of that same treeview. Why not just leave that bloody treeview there? Because I happen to be downloading Windows Vista in the background, it is taking about one minute to turn each page! (The Apple pages took a few seconds each to load.) Maybe it is deliberate that the documentation is impossible to navigate, to force developers to purchase the book with the same information? To hell with that resource.

The plan now is to learn about the new Windows Vista User Experience. With luck, I can get the Beta to work within Virtual PC, and I can see Vista in action. I’ll let you know how I go…

by Alister Jones | Next up: Charlie Seeks Friend, Open to Romance →

Taking Scissors to a Web Page

2006-03-30T18:01:00.000-08:00

In an attempt to find an interface style that would make Charlie easy to use, I first considered a conversational interface convention and then a PowerPoint interface convention. I very much liked the simplicity and user friendliness of the conversational interface convention, but a significant problem prevented me from saying, “That’s good enough. I’ll just run with that.”

The problem with the conversational and PowerPoint interface conventions is that they both see a webpage as being a single document. The user opens an entire webpage for editing, performs some edits, and then saves the entire document. The problem is that the webpage will be constructed from any number of separate plugins, and each plugin may need to present its own set of tools for the user to edit content. While it helps the user to see the webpage as a single document, it helps Charlie to see the webpage as separate pieces. There is a mismatch between the needs of the user and the needs of Charlie.

When I started thinking about the Windows Explorer and the Mac Finder, a solution occurred to me. Initially I thought that splitting a webpage into separate pieces (for the sake of Charlie) would cause confusion for the user. But then I thought about how users actually work with their computers. Office workers are used to working with Word, Excel, PowerPoint, and so on to create separate parts of documents. Designers are used to working with QuarkXPress, Photoshop, Illustrator, and so on to create separate parts of documents. The users employ Windows Explorer or the Mac Finder to keep the parts separate, and are used to firing up different programs to edit the parts separately. With the introduction of an explorer interface convention, website owners would be comfortable seeing the separate parts of their websites, and working on those parts separately, bringing the needs of the users in line with the needs of Charlie.

My only concern so far is that an explorer interface convention does not give Charlie a friendly face, which I have determined is important. But it may still be possible to integrate the best parts of the conversational interface convention with the new explorer interface convention. If the user fires up the Article Editor, the screen may look as follows.

When the user has finished composing or editing an article, he or she clicks on the Save and Publish command button. Charlie’s avatar can then report on what has occurred.

While the dummy screen above looks clumsy, I have not yet attempted to craft the interface. Still, I feel that I’m on the right path to an administration interface convention for Charlie.

by Alister Jones | Next up: I’ll Take What’s Behind Door Number Three →

An Explorer Interface Convention

2006-03-30T06:02:00.000-08:00

After reading through User Interface Design for Programmers by Joel Spolsky, I decided that I would give his CityDesk web creation program a whirl. While it has a few weak points, the program is delightfully easy to use.

The concept is simple. You have a treeview window that shows you files and folders, just like Windows Explorer. If you double-click on one of the documents an editor or viewer opens up, just like Windows Explorer. CityDesk is a desktop application for Windows; it is not a web-based application. Still, the simplicity, familiarity, and effectiveness of this approach got me to thinking about how it would work in a web-based application.

After scribbling a few ideas on paper, I fired up Photoshop. Here is the first screen I created. (All of these screens are dirt-simple, as I was just recording some ideas.)

Because this is such a familiar interface I am sure I don’t have to explain it. (I wouldn’t need to explain it to users, either.) But what will be transparent to users but of interest to us is that the folders with icons are not mere folders. Rather, they represent plugins. The Photos folder, when opened, will be handled by the Photos plugin. The Weblog folder, when opened, will be handled by the Weblog plugin. To test this idea, I next created the screen that the user would see if he or she opens the Photos folder.

The tasks area means that every plugin can provide different functionality, yet in a consistent way. Here is a further example of another plugin looking the same, but working differently. This example also shows the simple way in which draft documents can be handled.

If the user double-clicks on an Article document, the Article Editor opens and displays the document for editing, just as it would from Windows Explorer or the Mac Finder. What the following screen adds is the simple, text-only command bar. This would be added to the folder windows above, but I didn’t want to waste time reworking the above images.

There is a very important commercial benefit to using icons to represent functionality. Namely, the icons can be dimmed to show that the functionality is available but disabled, and the user must upgrade in order to enable that functionality. Alternatively, the dimmed icons can represent “demos”. A user with a basic-level membership can see the dimmed Blog icon, and can double-click it in order to create draft blog documents (remember, draft documents are dimmed too, so the consistency carries through). Creating draft documents allows the user to trial the additional functionality. Then, if he or she upgrades to a better membership level, the draft documents can be published.

Because I was getting a bit bored of Photoshop at this time, I threw together a quickie Control Panel in order to document the idea.

An explorer interface convention like this provides at least three main benefits. First, the convention is familiar to all computer users, so a user should be able to get going as quickly as possible. Second, the convention benefits from more than twenty years of user interface research performed by Apple and Microsoft. The third benefit is more involved, and I’ll chat about it in my next weblog entry.

by Alister Jones | Next up: Taking Scissors to a Web Page →

A Word from Joel - Part 3

2006-03-15T18:04:00.000-08:00

I’d like to comment on one final aspect of Joel Spolsky’s series of essays on User Interface Design for Programmers. At the end of Affordances and Metaphors, Joel explains that the tabbed dialog convention is incredibly successful in communicating with the user about how to transition between different option screens.

Why do I not adopt a tabbed interface convention? The following mockup, using the WordPress administration screen, shows why.

Firefox, Netscape, and soon Internet Explorer feature tabbed windows. The alisterjones.com website features tabbed navigation, which is a very popular navigation style. If Charlie’s administration screens adopt the same tabbed convention, then the user may end up viewing a screen featuring tabs within tabs within tabs. Users won’t be thanking me for that.

It is for this reason that I’d like for Charlie’s administration screens to clearly separate the browser from the contained website. This requires using an administration area that provides some visual space, and that features an interface convention that is different from the browser above and the contained webpage below.

While the above is still far from perfect, I do sense that the conversational interface convention has a lot to recommend it.

Still, my research on interface conventions is far from over.

by Alister Jones | Next up: An Explorer Interface Convention →

A PowerPoint Interface Convention

2006-03-14T18:05:00.000-08:00

When I first talked of my search for an interface convention, I mentioned that my original plan was to mimic Microsoft PowerPoint. I then dismissed this approach as being too limited. However, Joel Spolsky’s Consistency and Other Hobgoblins essay has made me reconsider this interface.

Here is the interface I came up with about 18 months ago, when I first started thinking about webpage editors.

To state the obvious, the benefit of this interface is that it is consistent with the Microsoft Office software with which most users are familiar. By adopting this interface convention I could, in one move, provide Charlie with an interface that behaves exactly how the user thinks it would, which is the definition of a good interface. Still, I do have concerns about this design.

First, a webpage is presented within a separate browser application. That means that the browser application will have its own set of drop down menus and toolbars, and then there’d be another set of drop down menus and toolbars within the child web application. This does not occur in any other desktop software, so such an interface convention is not as immediately familiar and externally consistent as it may first seem.

Second, I am not sure how far you could take this interface convention in a web application. The menu above includes an Insert Page option. What happens when the user clicks on it? The plan at the time was to present a set of page templates (such as Weblog Page, Article Page, Photo Gallery Page, and so on), exactly as PowerPoint does when you click on Insert Slide. That may be okay, but where in the site map would the new page appear? How would you give it a URL address? Can this interface support the concept of draft pages? While mimicking Microsoft PowerPoint provides the benefit of familiarity, I wonder whether it does so at the cost of simplicity and flexibility.

Still, there it is, another possible interface convention.

by Alister Jones | Next up: A Word from Joel - Part 3 →

A Word from Joel - Part 2

2006-03-12T18:06:00.000-08:00

In the second part of Joel Spolsky’s User Interface Design for Programmers series, a few more interesting ideas are raised.

The first idea is that users do not read manuals, and they do not read instructions. This was one of the forces behind my conversational interface convention. With only a few exceptions, it is not always clear what pressing a button will do. The “Edit” button edits what? Edits the whole page? Edits the photo above? Edits the text below? Given that users do not read manuals or instructions, how can we educate the user? My current resolution is to put the button within a sentence.

Placing a button within the context of a sentence supports both new users and frequent users. New users can skim the text and be sure of what that button will do. Frequent users will “tune out” the surrounding text and will just see the three clickable buttons. Sure, this takes up more vertical space than if three unadorned buttons were placed side by side. But a webpage is a document with virtually unlimited vertical space, unlike a desktop application where the screen space is limited. I feel that the user-friendliness of this approach is worth the price of the vertical space it requires.

A second interesting idea put forth is the following:

“An important rule of thumb is that user models aren’t very complex. When people have to guess how a program is going to work, they tend to guess simple things, rather than complicated things.”

If you’ve been following my ramblings for a while, you’ll know that I agree with this rule of thumb in the strongest possible way. But let me tell you a concern that I have. I am so astonished at the complexity that developers introduce to interfaces that I have started to wonder whether I am actually a bit stupid. There’s only two possible truths. Either 99% of developers are right—users accept complexity—and I’m wrong. Or, I’m right—users expect simplicity—and 99% of developers are wrong. If you think I’m being modest, I’m not. This truly worries me. So that I don’t fall into another sea of doubt, I’m going to presume that Joel is right, most other developers are wrong, and I should aim to make Charlie as simple to use as possible.

by Alister Jones | Next up: A PowerPoint Interface Convention →

A Word from Joel - Part 1

2006-03-10T18:07:00.000-08:00

I have come to the point in the development of Charlie where I need to decide upon an interface convention. Without researching the topic, a from-the-heart approach occurred to me which I have named a conversational interface convention. (By the way, I had to name it simply to give the weblog entry a title—not because I think it’s worthy of an official name.)

The plan now is to properly investigate interface conventions, to evaluate any conventions I find, and decide upon a convention for Charlie. I have decided to start with Joel Spolsky’s User Interface Design for Programmers. I love this guy’s writing, and I believe in what he writes about. If it’s not immediately apparent, this weblog is fashioned after Joel’s own website. The pictures of Marilyn that pepper my posts? Inspired by the “break this page up” photos that Joel adds to his posts. The poor attempts at humour? Inspired by Joel’s suggestion that technical writing need not be boring. The analogies to popular culture? Inspired by Joel’s own stories. I’ll never be as good a programmer or as interesting a writer as Joel, but I’m doing my best.

After a funny story about a skinny kid and a blob of dough in a bathtub on wheels (one of which is wonky), Joel arrives at the following golden rule of interface design.

“A user interface is well-designed when the program behaves exactly how the user thought it would.”

It would be easy to dismiss this rule as being so common-sensical that it does not warrant consideration. But you need only look at virtually every bit of software and every website to know that this golden rule is very rarely achieved. So let’s give this rule some consideration. Or, more accurately, let me give you my take on it.

It is my nature to try to reduce things to a few core ideas or values. My brain is wired in such a way that I cannot comprehend great lumps of information. But if I can break something down into smaller units, and see the logic that binds those units, then I can comprehend a complex topic. If you’re smarter than me, you probably don’t need to reduce Joel’s rule into smaller parts. But my brain can only dine on Corn Flakes.

It is my interpretation that a user interface will behave exactly how the user thought it would if it is both externally consistent and internally consistent.

A user interface is externally consistent if it behaves the same way as other programs. Jakob Nielsen promotes this idea very heavily and, while I don’t agree with everything he says, I reckon he’s right on the money about this. If you come up with a completely new interface paradigm, then you’re free to make your own rules. But if you are working within a known interface paradigm, then if behooves you to make your application externally consistent. I think most web applications get this very, very wrong, and I’ll look at specific examples when I review existing web applications.

A user interface is internally consistent if it keeps behaving the same way in response to the same user actions. If a similarly-styled hyperlink sometimes takes you to another page, sometimes submits a form, sometimes executes some JavaScript, sometimes pops open a new window, and sometimes scratches your nose, then the web application provides no internal consistency. Again, I’ll look at specific examples shortly.

So that’s the first lesson from Joel. But class is still in progress…

by Alister Jones | Next up: A Word from Joel - Part 2 →

A Conversational Interface Convention

2006-03-08T18:10:00.000-08:00

Over the last few days I have been thinking a lot about a possible interface convention. So far, I have come up with one possibility, which I will call a conversational interface convention. The idea is that Charlie talks to the user, and the user replies. Let me illustrate.

Here is what an administrator would see if she logs into my own website and navigates to an article page. The dog avatar is not what I would use, but I like the casual illustration style of Steve Barr. The avatar always provides feedback on what is happening. If an AJAX-type callback is in progress, the avatar would say something like, “Please wait while your updates are saved.” Following the speech bubble, the user can say what she would like to do in response to what the avatar has said.

If the user clicks the Edit button, the process moves into Edit mode and the user will see the following page.

While it’s not especially relevant to the topic at hand, I envisage using MarkDown.NET rather than a rich text editor, but I’ll talk about that at some later point. What is notable is that the speech bubble has been updated, and the user has new options. If the user updates the content, she clicks the Preview button. The process moves into Preview mode, and the user will see the following page.

What you may notice is that the process is presented in a Wizard-like back and forth series of pages, but without the limits of a true Wizard implementation. The same conversational interface convention extends to other parts of the administration pages, such as the Control Panel:

What I like about this interface convention is that it puts into action the two decisions made about the ghost in the machine. That is, Charlie is given a friendly face and is brought to the fore. (Again, I would not actually use a dog.) Additionally, each button press results in a consistent and predictable full-mode change. The ability to have the avatar provide feedback, and then show the user a list of possible responses, is very simple yet infinitely flexible. This interface convention could, I believe, be extended to every possible administration page.

This is just the first interface convention that has occurred to me. I felt that creating a rough draft in Photoshop would provide a “taste” of the idea, and allow my pending interface research to have a point of reference. Even better, I’d love to know what you guys think of the concept.

by Alister Jones | Next up: A Word from Joel - Part 1 →

Getting Real. Get It!

2006-03-07T18:11:00.000-08:00

I have finished reading Getting Real, and I recommend that you read it too. Whether you are a designer, a programmer, a project manager, or someone who found $19 on the street, you will get your money’s worth from this book.

As you will see from my recent weblog entries, I did not agree with all its recommendations. But that’s okay. The book challenged me to articulate my own ideas and my own beliefs, and that is a valuable exercise. Most importantly, the guys from 37signals have encouraged me to have faith in my ideas; they pulled me out of the sea of doubt that I had found myself in, and for that I will be forever grateful.

by Alister Jones | Next up: A Conversational Interface Convention →

One Interface or Two?

2006-03-06T18:12:00.000-08:00

The authors of Getting Real recommend that a web application provide just one interface. Any administration tasks should be incorporated into the same interface that the public sees. This is an implementation of the half-mode change that I discussed in the previous weblog entry. Well, I disagreed with the concept of a half-mode change, and I certainly disagree with the concept that a web application should provide just one interface.

To incorporate the administration screens in the public interface means to extend the administration plumbing out into the public areas, which is about as appropriate as extending the plumbing of a house out into its living rooms and placing a toilet beside the lounge. There are many reasons why a toilet belongs in a bathroom, and privacy is just one reason. There are mechanical and architectural reasons too.

It is my contention that a website requires exactly two interfaces, and that both interfaces are very, very different. One is the web site interface and the other is the web application interface.

The website interface is what the visitor sees. The website interface should be completely unaffected by any administration concerns. If the website displays surfboards, then it should be free to present the boards using HTML, Flash, SVG, PDF, or whatever technology best allows the website owner to communicate with its customers. If you attempt to shoehorn the administration stuff into the same interface, then you’re limiting the choices available. The goal of maintaining just one interface penalizes the website owner’s message and the visitor’s experience, for the benefit of the developer, and that’s just wrong.

The web application interface is what the website’s administrators see. The web application interface should be completely unaffected by the pages that the visitor sees. The administrators should be able to edit content, never needing to worry about whether that content will be served to a browser, to a PDA, to a cellular phone, to an RSS aggregator, to a web service SOAP response, to the Google spider, to a PDF document, or to any other type of device or document. Moveover, the administration screens should be able to use whatever trickery is necessary to support the administrators, such as JavaScript and AJAX and form processing, even though such trickery may not be permissible in creating an accessible website.

To put it another way, the web site should be free to use the simplest and most effective approach possible for the website’s owner to communicate with its customers. The web application should be free to use any amount of complexity necessary to carry out administration tasks. The web site and the web application have different purposes, different requirements, and different forces. What good can come from trying to mesh the two together?

Charlie will not attempt to achieve a single interface by incorporating administration functions in the public website. The public website and the private web application will be kept separate.

by Alister Jones | Next up: Getting Real. Get It! →

Should the Ghost in the Machine be Invisible?

2006-03-05T18:13:00.000-08:00

For anyone new to this weblog, I am developing an application named Charlie. At this point in time, I am seeking an interface convention to ensure that all administration screens act in a consistent and predictable way. To this end, I have been thinking a lot about interfaces and how users interact with software applications. Right now, I am considering two separate but related aspects of interface design. First, should Charlie itself be invisible to the user? Second, should Charlie’s mode changes be invisible to the user?

Now, should Charlie itself be invisible to the user? I have said earlier that software should stay out of the way while the user performs a task, but that is not the same thing as being invisible software. The original Mac operating system provides a great example. The OS never got in the way, but Apple did not hide it, either. To start with, the operating system had a face:

This is no mere gimmick. This is a message to the user: “I am here. Know me. Love me.” Apple did not hide the OS away; it brought it to the fore and made it a friend. Microsoft does not take this approach with its Windows operating system. The operating system is kept in the background, and makes no attempt to befriend the user. Both operating systems stay out of the way, but Windows walks behind the user, and the Mac walks beside the user. It is my opinion that the Mac approach brings with it two real benefits that I’d like for Charlie to exhibit too.

One benefit to making software a friend is that it enables the user to forgive the software if it displays any shortcomings. It is easy to get mad at Windows, because it operates like it has a bunch of goblins running about, out of sight, causing mischief. Windows is quite unapologetic about its screw ups. But it is hard to get mad at the Mac, because it operates like a friend who, on occasion, gets things a little wrong. The Mac will show you its face, tell you what’s gone wrong, and effectively says, “Look, I messed this one up. But we’re still friends, right?” Right!

Another benefit to making software a friend is that it makes the user reluctant to let it go. You just don’t discard friends. (Or if you do, I don’t want you as a friend or as a customer.) If the users of Charlie form an emotional attachment to the software, then they will not switch to another application just because it is a little cheaper or because it offers a few more features. Friends stick together.

The second aspect of interface design that is on my mind is whether Charlie’s mode changes should be invisible to the user. And what is a mode change? It is a switch to a different activity. (I don’t know the true definition of a mode change in software. And it’s not important anyway. What’s important is the principle being discussed, not the name applied.)

Say that the owner of Homunculus.com wants to edit his Carla Gugino page, because he wants to let the world know that she gets naked in Sin City. He opens his browser and navigates to Carla’s page. Right now, he’s in View mode. How should he switch to Edit mode?

If the change from View mode to Edit mode is invisible (which can almost be done), he’d simply place the cursor at the end of the page, and start typing the new content. If he navigates away, some Ajax trickery can send the updates to the server to be saved to the database. We might say that this is a zero-mode change.

If the change from View mode to Edit mode is explicit but occurs without fetching a new page (get a free account with BackPack to see this in action), he’d click on an “Edit this Content” button which replaces just a part of the page with a textbox. The website owner can add his comments about Carla’s curves and then click on a Save button. The textbox disappears and the new content is displayed (more Ajax trickery). Since a part of the page changes mode while the rest of the page stays the same, we might say this is a half-mode change.

The final main option is that the website owner can click on an “Edit this Content” button which fetches a whole new page which makes no attempt to look like the live page, and attempts only to present a form with which the owner can update his website. Since the entire page changes and there is an explicit switch between View mode and Edit mode, we might say that this is a full-mode change.

The question is, which type of mode change is best? Zero-mode, half-mode, or full-mode? The answer to this question means deciding just how transparent a mode change ought to be.

It is my belief that zero-mode changes are dumb. How many times have you been bothered by users who become completely confused because they mistakenly undocked a toolbar, or moved or resized the taskbar? The need to undock a toolbar or move a taskbar is so rare that it is dumb to make it a zero-mode-change operation; all it leads to is confused and annoyed users. Without an explicit mode change, the user has no idea where they are in the task they want to perform, what they can do at this point, and what they can do next.

It is also my belief that half-mode changes are an inadvisable goal for software. To start with, it can be disorienting for the user. There is no sense of “location within the task,” since a part of the screen is performing one task (showing content) and another part of the screen is performing another task (editing content). Another problem is that not all tasks can be undertaken with a half-mode change. That means that sometimes the Edit button will initiate a half-mode change, and sometimes the Edit button will initiate a full-mode change. This unpredictable behaviour provides for a poor user experience. A further problem is that such “in-place editing” provides no opportunity for draft content, because all changes are made to the live content. That’s okay for a narrow range of software, but for a website framework it is important that the owner can start to create a new page and then change his mind. If the website owner suddenly finds that he doesn’t have enough information to finish the content, then he should have the option to save the content in draft form, and finish it later.

It is my belief that full-mode changes are the only realistic target for an interface convention. By making full and explicit mode changes, the user will always know where he is within a given task, and what he can do next. Also, if the same full-mode change is made every time an Edit button is clicked, this will provide the user with consistent and predictable behaviour. Finally, full-mode changes allow for complete control over what to do with the data on the “edit this content” page; that data can be discarded, or saved in draft form, or published to the live website. Sure, full-mode changes are clunky and uncool, but they are flexible, predictable, and consistent.

Unless further investigation shows that my beliefs are mistaken, Charlie will be visible to users, and its administration pages will involve a full-mode change.

by Alister Jones | Next up: One Interface or Two? →

The Interface. The Chicken or the Egg?

2006-03-03T18:14:00.000-08:00

In my previous weblog entry, I disagreed with the advice in Getting Real about getting to “Done!” as soon as possible. In this entry, I will disagree with another piece of advice, which is that you should design the application’s interface before you start programming. Hogwash!

Every software application is created to solve a problem. Even a game is created to solve the problem of, “I’m bored.” Surely it stands to reason that if a software application is created to solve a problem, then the first thing you should do is solve the bleeding problem! You could design the sexiest interface in the world, but if it is not attached to software that solves the problem, then the software will fail. You could design the fastest database in the world, normalised to the tenth normal form, but if it is not attached to software that solves the problem, then the software will fail.

This is a common complaint about games released in the last few years. They look great, run fast, but are simply boring to play. The actual problem of, “I’m bored, entertain me,” has not been solved. In contrast, check out Boom Boom Volleyball. This game looks poor, runs slow, but is fun to play. Since it solves the stated problem, is it not successful?

Another problem with the interface first approach is that it will lead to shortcuts. The idea put forth in Getting Real is that you create the interface, first on paper and then in HTML, and then turn that static HTML into a dynamic application. This is the approach that leads to developers creating .aspx pages that grab data directly from the database, loop through that data, and spit it out onto the page. Sure, it works, but it is a shortcut between the data and the interface. What is missing is the business layer—the layer where the problem is solved.

A further problem with the interface first approach is that it leads to software that grows by addition, not by multiplication. (For the more mathetically inclined, the growth is linear, not exponential.) To add a new feature means to create a new static interface element, and then add all the code necessary to turn that static element into a dynamic element. To add another feature means to do the same thing again. What is missing is any chance for emergence to occur and to grow the software. If the developer spends time solving the problem in the business layer, and adds the infrastructure to support that solution, then other features can grow naturally from that base. Each time you expand that base and make it richer, the less effort needed to implement the next feature.

The second argument put forth by the authors of Getting Real for designing the interface first is that “the interface is your product.” Again, that’s hogwash. Nobody buys a software interface. Customers buy a solution to a problem. The interface has a huge part to play in whether the product delivers the solution, but it is the solution a customer buys, not the interface.

As an aside, I’ve never understood how “Which came first, the chicken or the egg?” qualifies as challenge. The egg came first. Unless evolution can occur within a single animal’s life—but it cannot—then the egg must have come first. Think about it.

by Alister Jones | Next up: Should the Ghost in the Machine be Invisible? →

The Power of Two

2006-03-03T16:15:00.000-08:00

So far I am enjoying reading Getting Real. The authors take a very pragmatic approach to most of the problems of software development, which is a refreshing change from the stringent approach taken by most authors. What is interesting is that Getting Real presents many of the same arguments as Agile evangelists, but without the zealous fervour of those evangelists. With the zealots, it is very hard to see passed bullshit and through to the ideas. With Getting Real, there is no bullshit, so the ideas are right there to be learned and applied. It is a great book and I recommend it highly.

What I cannot recommend, however, is the advice on the page entitled, “Done!” The idea presented on this page is that decisions are temporary, a poor decision is not the end of the world, so just make a decision and move on. I think this is poor advice.

Earlier in the same book, a quote is presented from The Pragmatic Programmer by Dave Thomas. The quote is as follows:

“As the designer or developer of a new application, you’re faced with hundreds of micro-decisions each and every day: blue or green? One table or two? Static or dynamic? Abort or recover? How do we make these decisions? If it’s something we recognize as being important, we might ask. The rest, we guess. And all that guessing builds up a kind of debt in our applications—an interconnected web of assumptions.”

I feel that if a developer accepts that all decisions are temporary and relatively unimportant, then the developer is just going to start making guesses so that he or she can get to “Done!” as quickly as possible. As Dave explains, this builds up a kind of debt in the application. And like all debts, it will become due for payment at some later date.

During the development of Charlie, I have noticed that I have evolved my own approach to making a decision. First, I will read just enough about the subject so that I understand the issues involved. Then, I will read how Microsoft recommends that a particular problem be solved. Next, I will put pencil to paper and see if I cannot come up with a simpler solution. Only after comparing Microsoft’s recommendation against my own ideas will I make my decision. That is, I end up comparing two recommendations: Microsoft’s and my own.

Throughout this weblog, you will have noticed that I often say, “There are two main approaches,” or “There are two possible solutions.” Often they are salt-and-pepper options; one will be direct and one will be indirect, one will be simple and one will be complex. Again, I end up comparing two approaches.

I feel that two is a good number. If you don’t evaluate at least two alternatives, then you’re really just relying on gut instinct, which is little more than an outright guess. Conversely, if you try to evaluate more than two alternatives, then you probably haven’t yet distilled the problem down to its core. At the core, most problems have two forces pulling in different directions: direct against flexible, simple against complex, developer against user, and so on. If an issue does involve more than two forces, I reckon the issue qualifies for being broken down further, again and again, until you end up with a set of easily-stated problems, each with just two main solutions. Then, you need to decide between the two. That will be a true decision, not a guess masquerading as a decision.

So this represents a different approach to that recommended in Getting Real. Rather than getting to “Done!” as quickly as possible, I will aim to get to “two” as quickly as possible, and then make a decision.

by Alister Jones | Next up: The Interface. The Chicken or the Egg? →

Opinions and Preferences

2006-03-03T13:16:00.000-08:00

In an episode of Seinfeld, we see the interaction between Jerry and his assistant. His assistant annoys him about the smallest little details; so much so that he sometimes forgets what he’s doing.

Software can be like that. A great example is provided by Joel Spolsky when he talks about the Find Setup Wizard.

Have you seen this dialog? Sure you have. What was your response? My initial response, and every response since, has been, “What the hell are you asking me? Why are you asking me this? What happens if I make a poor choice? Go away!”

To ensure that your own application does not become like Jerry’s annoying assistant, the authors of Getting Real recommend that you make opinionated software. An application should have its own opinions, and apply those opinions wherever possible, thereby leaving the user to concentrate on whatever it is he or she is doing. I think that’s sound advice.

Another section of the book recommends minimising the number of preferences that your application makes available to the user. Or in the words of the book, “Decide the little details so that your customers don’t have to.” This is another take on making opinionated software.

It would seem that to make software with more opinions and fewer preferences is to remove control from the user. But does it? Why is the user sitting at the computer in the first place? To mess around or to get something done? Remember that most users are not developers who will feel cheated if they cannot control every little aspect of the application. Most users just want to get something done without the software getting in the way. And that is not to say that most users do not care about the software. I don’t mean that at all. I mean that most users truly want the software to stay out of the way while they concentrate on the matter at hand.

One of the most powerful, most popular, and most productive software applications is Adobe Photoshop. With this software, users can do anything. Every magazine you read, every movie you watch, and every website you view has probably been touched by Photoshop. Yet if you look at the number of preferences it makes available, you will wonder if you mistakenly bought Photoshop Lite. If Adobe Photoshop can provide infinite flexibity with so few application preferences, you will see that opinionated software really does empower the user, not limit the user.

What does this mean for Charlie? That no preferences will be made available? No, not at all. Good software is flexible, and I’d like for Charlie to be deemed good software. What I will attempt however is to minimise the number of preferences that are exposed to the user. The flexibility will still be incorporated into Charlie, but I will set as many preferences as possible, leaving only the most important to be set by the user. In that way, I hope that Charlie will be the helpful assistant that Jerry truly wanted.

by Alister Jones | Next up: The Power of Two →

Love the Little Guys

2006-03-02T23:17:00.000-08:00

An important idea that Getting Real discusses (in just two paragraphs—that’s how simple the book is) is to find the right customers. The corollary is that if you try to please everyone, you will not please anyone.

While that is well-known wisdom, how often is it forgotten in designing software? Because I have chosen DotNetNuke as Charlie’s enemy, I will use this software as an example. DotNetNuke attempts to be all things to all people. Its core and its modules support the development of small websites and the development of large websites. If you view the forums for this software, you will see that it does not provide the simplicity for small websites nor the flexibility for large websites. It attempts to please everyone, and ends up pleasing no-one. Well, the only people it pleases are gimmick-loving developers, but I have already argued that that is the wrong target for a website framework application.

Who will Charlie target? Charlie will target the websites created for individuals and small businesses.

The first and most important reason that Charlie will support individuals and small businesses is that this is what I enjoy. I enjoy having a small part to play in the individual’s life or the small business’s growth. If I were to target large companies, I would be a dispensable contractor, and where’s the fun in that?

The second reason that Charlie will support individuals and small businesses is that this is a market with little support. For large companies, DotNetNuke or Community Server would fit the bill. But both applications are overkill for the little guys.

But even a market description of individuals and small businesses is too broad a definition. Charlie would need to be an amazingly complex application to support all individuals and all small businesses. So, what is the subset of this market that Charlie will target? I have chosen two initial markets.

The first market is photographers. I have one personal reason and one professional reason for targetting photographers. The personal reason is that I adore photography. I find it such a beautiful form of art that it pains me to see some of the awful websites used to present such beauty. I want to do it better, for the sake of the photographer. The professional reason that I want to target photographers is that photography represents one of the hardest parts of a website framework. Any noodle can create an interface by which a website owner can add and edit the text of the webpage. But how to create an interface by which a website owner can upload an image, work with that image, and place that image on the webpage? I feel that if I can solve that challenge, the resulting interface convention can then be applied to both simple and complex administration challenges.

The second market is wine makers. Once again, I have one personal reason and one professional reason. The personal reason is that many wine makers are individuals who have a vineyard as a business on the side. These are the sorts of small businesses that I enjoy working with. The professional reason is that to provide a website framework for wine makers requires proper business objects including a bottle of wine and its tasting notes. I want Charlie to provide the wine maker with the option to “Add a new wine product,” and then properly present an interface by which the wine maker can add the product’s title, volume, bottling date, acidity, and so on. This is in stark contrast to giving the wine maker a rich text editor and telling him, “Type whatever you like.” This is where I see that I can make Charlie different from the rest. Most website framework applications effectively say to the website owner, “Look, we don’t give a shit what your business is about. We’ll just provide you with some tools so that you can write whatever you like.” I think that sucks, so Charlie is going to care about the business of the website owner.

After Charlie has successfully supported photographers and wine makers, it will be evolved to support other individuals and small business. Charlie will love the little guys.

by Alister Jones | Next up: Opinions and Preferences →

Charlie’s Enemy

2006-03-02T22:18:00.000-08:00

At this stage in the life of Charlie, I am following the teachings of Getting Real. The book advises that a new software application should have an enemy. The book states, “Sometimes the best way to know what your app should be is to know what it shouldn’t be.”

Throughout this weblog I have picked on DotNetNuke like it was a snot-nosed little kid who put his pants on backwards. Technically, DotNetNuke is a cleverly architected application that many, many developers use to earn good income. Moreover, DotNetNuke will forever be more advanced than Charlie could ever be. If anything, Charlie will be the 90-pound weakling who will have sand kicked in its face by the mighty DotNetNuke. So why do I use DotNetNuke as Charlie’s stated enemy?

The reason I dislike DotNetNuke so much is that it is software created by developers for developers. That is wrong. We developers, we’re just middlemen. Our task is to provide a medium through which a website owner can talk to its audience (which may be friends, business partners, customers, or others). A website framework should target one or both of those principals—the owner or its audience—and not the middleman.

To provide an example, have a look at the following web address:

www.DotNetNuke.com/About/ProjectOwner/tabid/822/Default.aspx

That web address is clearly created by developer for developers. It has crap in the address that is designed to support developers, but that penalizes the website’s owner and the website’s audience. Such a long address cannot be added to the owner’s marketing material, and cannot be easily described over the phone. Such a long address cannot be easily typed by a visitor. If the software was created for website owners and its audience, that sort of crap would never find its way into the software.

If a gimmick comes out of Microsoft, it is immediately added to DotNetNuke. A Data Access Application Block was released by Microsoft and—slap!—in it goes to DotNetNuke. The new concept of Themes was released by Microsoft and—bang!—in it goes to DotNetNuke. What does a website owner care if the site is styled by CSS, a Theme, or a monkey with a paintbrush? The only reason this stuff goes into DotNetNuke is to appease developers; it is software created by developers for developers. To my mind, a feature should be added to an website application only if it adds value to the website’s owner or the website’s audience.

So Charlie has found its enemy.

by Alister Jones | Next up: Love the Little Guys →

In Search of an Interface Convention

2006-03-02T21:19:00.000-08:00

Those of you who have followed this blog for a little while will know that I recently stated that Charlie was on track, and was successfully serving simple, secure, localized, and configurable webpages. I then stated that I was moving on to design the administration interface. So what happened? Why am I suddenly stumbling about like a lost soul or a drunken teenager?

The reason is that I am in search of an interface convention. Not a code convention, but a presentation convention. I will nonetheless use a code convention to illustrate the problem. A popular code convention is the Model-View-Controller design pattern. This is an extremely versatile convention that works for simple interfaces and for complex interfaces. If you introduce this code convention into your Interface layer, it is unlikely you’ll ever need to replace it with something else. Similarly, I am seeking a presentation convention that works for simple interfaces and for complex interfaces, making it unlikely that I’ll ever need to replace it with something else.

My initial plan with Charlie was to create an interface that mimicked Microsoft PowerPoint. Say what you will about this program, it enables even the most technophobic person to create a slideshow presentation. Everyone can use this program without ever having to read its instructions. That’s good software. I felt that the slide-by-slide nature of PowerPoint mapped neatly to the page-by-page nature of a website. I also saw other analogies between the two. The problem with this approach is that it would be fine for a simple text-based website, but it would not scale to anything else. It does not provide a convention that would work for simple interfaces and for complex interfaces.

I am desperately keen to avoid a situation where to edit the text of a website the owner “edits in place” (like PowerPoint), and to edit the graphic of a website the owner needs to “open a new window”, and to change the settings of the website the owner needs to “use the control panel”. Such a situation would mean that the software behaves differently depending on what button the owner has pressed. This mutating behaviour means that the website owner must come to terms with the software’s quirks, rather than using the software as though it were not even there. In describing usability, Joel Spolsky tells us that, “Something is usable if it behaves exactly as expected.” Interface consistency—that is, an interface convention—has a large part to play in whether software behaves exactly as expected.

The development of Charlie has been put on hold while I search for an interface convention.

by Alister Jones | Next up: Charlie’s Enemy →

The Secret of Life - Continued

2006-03-02T20:20:00.000-08:00

In my last weblog entry, I opened with some advice presented in the movie City Slickers. I then waffled on about what might be important, without coming to any conclusion.

Well, another line from the same movie has Curly saying to Mitch, “You city folk, you sure worry about a lot of crap, don’t you?” (Something like that, anyhow.) That’s me, worrying about a lot of crap.

In a draft of a weblog entry that never got published, I said that DotNetNuke is complex software that produces ugly websites. I then said that Charlie is to be simple software that produces beautiful websites. Well, that’s good enough. It defines what Charlie is—simple software—and what it does—produce beautiful websites. That’s good enough.

by Alister Jones | Next up: In Search of an Interface Convention →

The Secret of Life

2006-03-02T19:21:00.000-08:00

In the wonderful movie City Slickers, Mitch is lost in his life, and takes a cattle-driving vacation in order to find some meaning. Curly takes Mitch into his confidence and explains that the secret to life is just one thing. Mitch desperately wants to know—needs to know—what is the one thing that is the secret to life. Curly tells Mitch that it is up to him to figure it out himself. Not because it is inexplicable, but because it is personal. The secret to life will be different for each and every person, but for everyone it will be just one thing.

At the start of Getting Real, the authors advise that a new software application should aim to do less, not more, than its competitors in order to gain an advantage.

The advice in the movie and the advice in the book really say the same thing: ignore what’s unimportant, and find what’s important. Find what is important, get it right, and your life or your application will be successful.

I have given this quite a bit of thought, and I have asked myself what is the one thing that Charlie should do? Looking at the elevator pitch and the feature list, there is no identifiable driving force. I have not discovered the one thing that is the secret to Charlie’s life. If I wanted an application that allows a website owner to edit his or her own website, I could have chosen DotNetNuke. But the fact that I have avoided DotNetNuke means that while my head doesn’t know what that one thing is, my heart knows that DotNetNuke will not satisfy it. The elevator pitch and the feature list were products of my head, and DotNetNuke would have satisfied them. So, what did my heart want?

My heart wants for my business, Edition3, and my product, Charlie, to help others achieve what they want. That may sound like a sales pitch. I guess that it is a sales pitch, but it is not an empty one. Dale Carnegie has said that if you want to succeed, help others to succeed. Well I do want to succeed, and I do want to help others to succeed. How can Charlie help?

Let me tell you that I have just spent two hours trying to answer the question. The truth is, I don’t yet know. Many ideas occurred to me, and I started explaining each one. But each one felt a little bit wrong. My heart knows that I can create an application to help others to succeed (even if just in a small aspect of their lives). But my head cannot yet articulate how that can happen. So I will work my way through Getting Real, and will use this weblog as a way of documenting what I discover along the way.

If you think that all of my doubt and worry and indecision means that I should not be doing this, you are probably right. But another lesson from Dale Carnegie is, “Don't give up.”

by Alister Jones | Next up: The Secret of Life - Continued →

When the Student is Ready…

2006-03-02T18:22:00.000-08:00

…the Teacher will Appear.

I was drowning in a sea of doubt. I was flailing about, trying to find a guide. The problem was, I was looking in the wrong direction.

If you look about the software development landscape, you will see a crowd of nerds standing on the hill, chanting like possessed men. “UML, UML, UML,” intone the nerds around their disciple Scott Ambler. “Charts, charts, charts,” hum the nerds around their disciple Craig Larman. “Statistics, statistics, statistics,” recite the nerds around their disciple Jakob Nielsen. Walk into that crowd of zealots, and they will pound you over the head with the heavy books they carry.

If you turn your back on the crowd of nerds, you will see a few guys sitting quietly beneath a tree. They don’t chant. They don’t beckon you over. But if you go to them, they will provide you with words of wisdom. They will talk from their heart, and not from their head. They will not force their own ideas upon you, but will encourage you to have faith in your own ideas, and show you ways to bring those ideas to fruition.

I was ready for these teachers. I had my own ideas, but the zealets kept telling me I was wrong, and I fell into that sea of doubt. I would never have seen these teachers if it were not for Milan Negovan pointing me in their direction. The teachers are the guys at 37signals. Their blessed teachings are passed on in their little book, Getting Real.

What makes these teachers so great? I desperately want to say, “They keep their message simple,” but I think if I mention the s-word one more time you will all scream. Instead, I will demonstrate by using a couple of icons of popular culture.

Sergio Aragones is an artist who can tell a story in a single tiny drawing. He’s the guy who creates the little illustrations in the margins of Mad Magazine. He can tell a funnier story in one basic drawing than most writers could tell in a whole page of prose.

Bruce Springsteen is an artist who can tell a rich story in a few verses. In a single line he tells of the passage of time:

But now there’s wrinkles around my baby’s eyes

If you’ve ever read a book by Jean Auel, you will know that many authors need a whole page to say the same thing.

Well, the guys from 37signals are in the same league as Sergio and Bruce. They can say in one sentence what would take Scott Ambler and the other disciples on the hill an entire page to say. (By the way, I am not unaware that these rambling weblog entries mean that I, too, am just as verbose.)

As I write this, I have only read a few dozen pages of Getting Real. But already it has told me to have faith in my ideas, and to not listen to the fools on the hill. Starting with the next weblog entry, I am going to work through this book, and apply its wisdom to Charlie.

by Alister Jones | Next up: The Secret of Life →

Lesson: Show Simplicity, Hide Complexity

2006-03-02T17:40:00.000-08:00

I know, I know, I rabbit on about simplicity as though it were the only thing of importance. Because I rabbit on, you’d think that you could look at my codebase and marvel at the tiny number of files and the small number of lines of code. However, if you were to view Charlie’s codebase, you’d see just as many files and lines of code as you’d find in a complex software application. The fact is that any non-trivial software application will require code, and lots of it. Where is the simplicity?

Simplicity comes from avoiding unnecessary complexity, and hiding necessary complexity.

First, simplicity comes from avoiding unnecessary complexity. And what is unnecessary complexity? It is complexity that is there “just in case” you might need it, even if the chance of needing it is about as likely as Kermit the Frog ever realizing the love of Miss Piggy. When I was creating the security infrastructure for Charlie, I had to consider anonymous users. Should I implement the equivalent of ASP.NET 2.0’s Profile object, allowing Charlie to store information about anonymous users, just in case I ever found the need? No. That would be unnecessary complexity. When I was creating the ability for Charlie to host multiple websites from a single installation, I had to consider the possibility of “mini-sites”—a website within a website, like the child portals in DotNetNuke. Should I provide the infrastructure for child websites, just in case I ever found the need? No. That would be unnecessary complexity.

Second, simplicity comes from hiding necessary complexity. And what is necessary complexity? As much as I’d like to come up with an impressive-sounding definition, I think necessary complexity is anything that makes you groan and say, “Fuck, there’s no simpler way to solve this.” While this complexity is necessary, it can still be hidden away in the depths of the application. In my last weblog entry, I described how I was able to add security and globalization to Charlie by tweaking the deep-down Entity System. Here is the bulk of the Article business object before I added security or globalization to Charlie (and using fields rather than properties, just to keep the code sample simple):

public class Article : Entity
{
    public String Title;
    public String Summary;
    public String Content;
}

I then added the security infrastructure to Charlie, and here is how the Article business object had to be refined:

public class Article : Entity
{
    public String Title;
    public String Summary;
    public String Content;
}

I then added the globalization infrastructure to Charlie, and here is how the Article business object had to be refined:

public class Article : Entity
{
    public String Title;
    public String Summary;
    public String Content;
}

Unless my cut-and-paste skills are hobbled, you’ll see that the business object never changed. The hundreds of lines of code needed for security and globalization is necessary complexity, but it is hidden away in the base Entity System.

Sure, this is a simple application of fundamental object-oriented programming, but that is exactly my point. If you take the time to apply simple OOP concepts, the resulting base classes provide fertile soil for your application. From that soil solutions can emerge, which was the topic of the last weblog entry. Into that soil complexity can be buried, which is the topic of this weblog entry.

The lesson is when you discover necessary complexity, see if you can hide that complexity. Bury it in the soil of your application.

by Alister Jones | Next up: When the Student is Ready… →

Lesson: Objects Allow Emergence, RAD Does Not

2006-03-02T17:24:00.000-08:00

By RAD, I mean Rapid Application Development. And by that, I mean the drag-and-drop-and-think-no-more approach taken by many users of Visual Studio. Need a navigation bar for your website? Drag and drop an XML-based sitemap source onto your page, drag and drop a Menu control onto your page, bind the two together, and think no more about it.

By emergence, I mean the concept put forth by Andrew Hunt. An emergent property is one that grows out of a base that was not intended to provide that property. If you look at your backyard you will, I hope, see some lawn. The intention is that it provides a nice green patch that the kids can run on and, if they fall over, not hurt themselves. But if you drop in a seed, a flower will emerge. If you drop in an acorn, a whopping great tree will emerge. The flower and the tree were not intentional properties of the lawn, but they evolved because the lawn provided a fertile base. (This is my own interpretation of Andrew Hunt’s use of the term. If this analogy does not match what Andrew meant, then the error is all mine.)

Let’s return to the example of the XML-based sitemap source used by the RAD developer. This solution requires a SiteMapDataSource control in the Interface layer, a SiteMapNodeCollection in the Business layer, a SiteMapProvider in the Data Access layer, and an XML source file. So here is the solution:

Next up, the RAD developer needs a bit of localized text. .NET accommodates the developer by providing a ResourceManager that manages separate Resource files, each of which is drawn from satellite assemblies. Here is the solution:

Finally, the RAD developer needs to store some information about the current user of the website. .NET provides a Profile object, with its own Provider, and stores the information in a self-configured SQL 2005 database. Here is the solution:

Each of these is a forced solution, designed to support RAD developers. They are designed separately and differently, they are implemented separately and differently, and they are used separately and differently. Why does localization use a Manager when the others do not? Why does localization not use a Provider, when the others do? Why does one use an XML store, one use an assembly store, and one use a database store? And I have provided just three examples. Consider the variety of separate and different approaches across the full ASP.NET framework. (I’m sure there’s a tautology in that last sentence, but I don’t think any English lecturers read this blog.)

The point is that none of these solutions emerged naturally. They are each banged in place to support RAD developers, with no rhyme or rhythm connecting them.

When I designed the Entity System for Charlie, I did so only because I had read Expert C# Business Objects, and I kind of liked the idea of a few base classes on top of which I could create my business objects. In other words, I went to the effort of creating the Entity System just so that I could be lazy later, and do a minimum amount of work to create business objects. That was the sole motivation for the Entity System.

What is interesting, however, is that this Entity System allowed other solutions to emerge—solutions that the Entity System was not designed to provide. When it came time to globalize Charlie, I did not have to implement all the separate pieces shown above. Rather, I just added a Culture property to the base Entity class, added a Culture property to the base EntityCriteria class, and added a Culture parameter to the base EntityMapper class.

When it came time to provide security, I did not have to implement all of the separate pieces shown above. I simply gave the base Entity class a set of CreateRoles, RetrieveRoles, UpdateRoles, and DeleteRoles, and told the EntityManager class to check whether the current user was in the appropriate role collection.

Most recently, I had to implement a sitemap for Charlie. I looked into the sitemap system in ASP.NET version 2.0, and ended up thinking, “To hell with that.” I already had a DocumentCollection class based on the EntityCollection class. I simply created a new DocumentMap entity, and gave it a property that exposed a DocumentCollection. And with that, I was pretty much done.

Before you think that I am patting myself on the back, I’m not. The Entity System was a simple thing created so that I could be lazy. Yet because it was simple and flexible, solutions emerged from its fertile soil. No foresight on my part—the emergence of these solutions has surprised me.

The lesson for the day, then, is that a little bit of object-oriented design leads to a surprising amount of emergence. Feel free to ignore ASP.NET’s own complex solutions, and let simple solutions emerge from your own codebase.

by Alister Jones | Next up: Lesson: Show Simplicity, Hide Complexity →

A New Day Dawns

2006-03-02T16:26:00.000-08:00

I’m tempted to remove the previous weblog entry and pretend it never happened. But that would be a lie, and I do not want to lie. The truth is that I had become drowned in a sea of doubt. I had spent nights doubting whether I was doing the right thing with Charlie. I will explain my doubts, not to put my heart on my sleeve, but to put Charlie in its context.

It is well known that to create a good product, in any industry, you must do the right thing, and do the thing right. Sure enough, I had doubts about both aspects of Charlie. Was I doing the right thing? Was I doing the thing right?

First, was I doing the right thing? Why was I creating a new website framework? If I create a simple website framework, it will forever lag behind the likes of DotNetNuke, Community Server, WilsonWebPortal, and others. I have said before that I hate DotNetNuke, and that has not changed. But what about the others?

Community Server is created by some of the smartest cookies in the ASP.NET jar, and it has been used to create some truly impressive websites. I have the strongest sense that I should be using it and, to use a cliché, stand on the shoulders of giants. The problem with Community Server is that it failed my litmus test. Specifically, I could not understand its licensing, I could not find where to download it, and I could not understand whether I could extend it and, if so, how. My litmus test says that if a product is presented in such a way that I cannot shake its hand, then there’s little chance of me being able to work with it. Community Server has gone into the too-hard basket.

The Wilson WebPortal is a release I have been looking forward to ever since I hear a whisper of it. Paul Wilson is a guy for whom I have an enormous amount of respect. He is very smart, but softens his intellect with a healthy dose of modesty. He respects simplicity and eschews needless complexity—forgive the redundancy. And he is very open, providing heaps of source-code components for anyone who spends just $50 to subscribe to his website. That’s not a sales pitch, but if you look at what others charge for crappy .NET stuff, you appreciate Paul’s openness and willingness to share. The reason I am not keen on using Wilson Web Portal is that it is very ASP.NET 2.0-centric, and I have grave concerns about ASP.NET version 2.0, which I will address next.

Let me tell you that I love ASP.NET version 1.1. The fact that I can pick up ASP.NET, with no real experience in programming, and make the technology sing and dance, is a testament to its power and simplicity. (To start with, it seems a complex technology, but once you see through to its inherent design, it is a thing of simple beauty.)

But it seems that the current version of ASP.NET was created by Microsoft pinning ASP.NET version 1.1 to the wall, and inviting all its engineers to throw things at it. If any logic went into its design, the logic is opaque to me. If there’s any consistency in the new framework, that too is opaque to me. The poles of simplicity that supported ASP.NET version 1.1 have been broken under the weight of the gimmicks added to ASP.NET version 2.0. It is a bloated and unattractive thing.

Now it can be argued that if I don’t understand ASP.NET version 2.0, then maybe the problem is with me, and not with ASP.NET. Believe me, that is one of the things that has contributed to my doubts about Charlie. Maybe I am simply not smart enough to work with ASP.NET. Maybe Java Server Pages, or PHP, or Ruby on Rails, would be more agreeable to this dumb koala bear. But I do like ASP.NET version 1.1 very much, and I understand it quite well, so there seems to be no immediate gain to switching to another technology and facing the same bloody learning curve that I have just finished climbing.

Still, the fact is that I am simply not comfortable with the direction that ASP.NET is taking. I have been tempted to target Mono and not .NET. But that seems to be another example of Charlie Brown’s philosophy: the secret to life is to replace one worry with another. I am very tempted to look at Ruby on Rails, but that too seems to suffer the same philosophical argument.

Do I have a current resolution? Nope! So I’ll just move on to the second problem: am I doing the thing right?

This whole weblog has been following my attempts to do the thing right. Have I succeeded? I think so. I have so far met the main goals I set forth in my software specifications; namely that Charlie is simple, secure, globalized, and configurable. As the running software meets the initial software specifications, then it would seem that I am doing the thing right.

So there you have it. I think I am doing the thing right, but I don't know whether I am doing the right thing.

Does that largely explain the preceeding, and very negative, weblog entry? If so, I am going to carry on and finish telling you of the lessons I have learned so far. Then, I am going to tell you of the teacher who has come to my rescue and lifted me out of this sea of doubt.

by Alister Jones | Next up: Lesson: Objects Allow Emergence, RAD Does Not →

Silently into the Night

2006-03-02T15:28:00.000-08:00

I have spent a few nights trying to explain why I am giving up on this weblog, and maybe on Charlie too. But I cannot find the words.

Charlie and I wish you the health and happiness that we cannot find.

by Alister Jones | Next up: A New Day Dawns →

Lesson:A Dummy’s Approach to Agile Development(or, “If You Can’t Draw It, Don’t Code It”)

2006-03-02T14:28:00.000-08:00

I have an admission that I must make. While I have criticised authors like Scott Ambler and Craig Larman for pushing Agile development so forcefully in their books, I must admit that they are on to something. The problem, in my opinion, is that these authors present Agile development in such a monstrously complex way that it would only make sense to someone who has already used that development process.

Let me provide an example. Please imagine that motorbike racer Valentino Rossi has thundered down a racetrack’s straight, and is about to turn into the first corner. From Valentino’s perspective, the corner goes to the right. Which way does he turn the handlebars on his bike in order to turn right?

Let’s ask a nerd. “The front wheel of a motorbike at speed is a gyroscope that will roll about the output axis in opposition to the input axis.” So there you have the right answer. However, that description only makes sense to someone who has already experienced turning a motorcycle travelling at speed. For anyone who has not had that experience, the description is meaningless.

Let’s ask a teacher the same question. “A motorbike at speed is carrying a lot of momentum and, once it is going fast in a straight line, it will be very difficult to change its direction. Still, let’s say that Arnold Schwarzenegger is riding the bike, and has the physical strength to overcome the bike’s resistence to turning. Now, let’s say that he turns the handlebars to the right in precisely the same way as if he were on a pushbike travelling at 5kph. If he were on a pushbike, he’d turn the handlebars to the right. But on a motorbike weighing 150kg and travelling at 150kph, the front wheel (20kg) of the bike would go momentarily to the right, while the rest of the bike (130kg plus rider) will continue travelling straight ahead. The bike would literally trip over its own front wheel, just as surely as if someone had stuck a pipe through its spokes. Because the front wheel has moved slightly to the right, the bike would fall onto its left side, and Arnold will be nursing bruises for two weeks. What this means is that if the motorbike’s handlebars are turned to the right, the bike will fall to its left. If the handlebars are turned abruptly (as you would do on a pushbike), this will result in a crash. But if the handlebars are turned gently to the right, the bike will start to fall away gently to the left. Now, if the center of gravity of the bike is slightly to the left of its wheels as a result of this gentle falling, then the centrifugal force will mean that the bike steers to the left. The bike is literally falling gently into the corner. Hence, when Valentino approaches that right-hand corner, he will be push the right handlebar gently forward, which will momentarily turn the front wheel to the left, which will result in the bike falling to its right, and falling into the right-hand corder."

What does this illustrate? It illustrates that I have chosen a very poor example. But the point I was trying to make is that if you ask a nerd to explain something, you might get a technically accurate answer, but that answer will only make sense to someone who already understands the answer. To everyone else, it will be gobbledygook. To those of us who come into the software development profession from outside, and who therefore have no experience of any software development process, the technical descriptions of Agile development have us asking, “What the bloody hell are you on about?”

For those of you in the audience who will be nodding your head and thinking, “Thank God it’s not just me who doesn’t understand what those nuts are on about,” let me provide a Dummy’s Approach to Agile Development. This approach has the informal description, “If you can’t draw it, don’t code it.”

The Dummy’s Approach to Agile Development says that you should write down, in one sentence, what you need to do—what problem you need to solve. Examples from Charlie’s development would include, “Come up with a way to add data to a business object,” or, “Come up with a way to ensure only some users can view certain pages,” or “Come up with a way to present the same page but in multiple languages.”

The next step in this Dummy’s Approach is to research the topic a little, just so that you know what you’re up against. You don’t have to become an expert on the matter, but just know enough that you understand the basic issues.

Once you understand the basic issues, you can then break the problem down into smaller pieces. If you look back at my discussion of globalization, you will see that the basic problem of how to globalize a website breaks down to two smaller problems: find a way to get text out of the code and store it ‘somewhere else’, and then find a way for that text to be ‘brought back in’ when the code is running.

Break the problem down into as many small pieces as you can. For example, the problem for text to be ‘brought back in’ to the running application can be broken down into two further, but smaller, problems: first we have to ‘get’ the text that is stored ‘somewhere else’, and then we have to ‘sort out’ which bit of text to use.

Once you have broken the problem down into small pieces, take a piece of paper, and draw one square for each problem. Put each square anywhere on the page, it does not matter where. What that square represents is the class that solves that particular problem. This is a very simple relationship. For each little problem you have discovered, you will have one class that solves that problem.

Next up, give each square a name that looks a little bit like an American Indian name in Noun-Verb format. For this globalization example, names might be “TextSaver,” “TextGetter,” and “TextSorter.”

With a new sheet of paper, place the squares in some arrangement that shows which squares talk to which other squares. If one square needs to talk to another square, draw a line between those two squares. Work with this arrangement until you minimise the number of joining lines.

When you have arrived at an arrangement that looks good to you, consider how these different squares will talk to each other. Do they pass simple messages (“Give me the text for a SaveButton”) or do they pass complex messages (“For an authorized user that is in the Administrator role, give me each of the pages that he can edit”)? If the message is complex, draw a circle that will represent the passed message. Consider this to be like the ball that is passed between players in a game of sport. The ball will end up being another class in your application.

You may be thinking, “Well, that sounds easy enough when you’ve made up an example, but real-life is not that easy.” You are perfectly correct. The first diagram is almost sure to be a rough guess. If you have a few doubts about what the diagram should look like, or what squares and circles you might need, then go back and research a little more. Even if you re-read the same stuff, you will now have a reference diagram so that you can see where new pieces might fit.

The idea is that you read a bit, draw a bit, read a bit more, draw a bit more, back and forth until you arrive at a diagram that feels “right”. It doesn’t even matter if the diagram is right or not. What matters is that it feels right to you, and that it provides a map of what you actually need to code. Did I really just say that it does not matter if the diagram is right or not? Yes. You see, there is no such thing as “perfect” code, so it’s pointless to strive for it. Without perfect code, there’s only two types of code: crap code and good code. This Dummy’s Approach is a way to ensure that ours is always good code, and never crap code. Unless you’re involved in launching rockets to the moon or controlling medical apparatus, then good code is good enough. The time it would take to move the good code to excellent code would be time better spent with your family or friends.

Once you have your final diagram, you can fire up your text editor or IDE and actually create the code for those classes.

Is this a long way to go about creating code? No! It is the short way. If you follow this process, then you will know precisely what classes you need, and what those classes must do—so you will code by knowing. If you do not follow this process, you must guess at the first class you need, then have a guess at the second class you might need, then have a guess at the third class you might need, then change the first class because it is wrong, then remove the second class because it is now useless, and so on—you will code by guessing. In the process of creating Charlie, I have used both approaches. I can tell you honestly that the components I first illustrated on paper are still in Charlie. The components where I just “had a go” have been ripped out of Charlie. I cannot overstate the importance of describing the problem, then researching and drawing, researching and drawing, back and forth, and then, only then, finally coding.

Is this really based on the Agile approach? Who cares! Let me just say that this approach has been the basis of everything that has gone right with Charlie. Research and draw. Research and draw. And if you can’t yet draw it, don’t yet code it. Research more and draw more. When you can draw it, then you can code it.

by Alister Jones | Next up: Silently into the Night →

Lesson: Don’t Forget .NET

2006-03-02T13:29:00.000-08:00

In an earlier weblog entry, I noted that I get frustrated by writers who discuss only the “how” of software programming, without discussing the associated “why” of software development. To redress this—if only in the little world of Charlie—I am trying to discuss why I have made certain design decisions, and the consequences and limitations of those decisions. Am I doing any better than the writers I have implicity criticised? I have no idea, since only one person responded to my request for feedback (God bless you, Garbin). Even so, I’ll persist with my efforts to talk about development from a slightly different perspective.

Well, another thing that frustrates me in reading about software development is the common feeling that, “Boy, I must be dumb.” I may read an article where a developer provides a little component, and then describes that component. If the article is written by a first-rate developer, the code looks so clean and so effective that I end up feeling a bit depressed—feeling sure that I’m simply not smart enough to ever write such good code.

The thought that gets me beyond such self-doubt is the image of a graceful duck. To a person standing on the shore of a pond, the duck appears to glide gracefully and effortlessly across the surface of the pond. What the observer does not see is that, below the surface, the duck is paddling like mad. It helps me to think that below the surface of code presented by a first-rate developer, the developer had to paddle like mad to arrive at that final code.

While I am not a first-rate developer, I’d still like to share some of the mad paddling that has gone into the development of Charlie. Specifically, I’d like to share some of the mistakes I made, and the lessons learned from those mistakes.

In developing Charlie’s architecture, I did a smart thing and included in the diagram a Foundation layer, which would comprise the .NET Framework Class Library. That smart step was, sure enough, followed by the dumb step of completely forgetting that foundation.

In coming up with the implementation of the Presenter object, I had to decide what types would be used to represent the Model, the View, and the Controller objects that the Presenter brings together. The View was easy, as I had created a new base View class that inherits from ASP.NET’s UserControl class. The Controller was easy too, because I just created a new base Controller class. But the Model had me stumped. Sometimes the Model would be a single Entity, and other times it would be an EntityCollection (refer to the Entity System). The problem to be solved was what type could the Presenter use to represent the Model? To illustrate, here is the relevant code from the Presenter class:

public abstract class Presenter
{
    public abstract ????? LoadModel(Int32 id);
    public abstract View GetView(ViewMode viewMode);
    public abstract Controller GetController(ViewMode viewMode);
}

A concrete Presenter can easily implement the GetView method, since I had come up with a base View class. The concrete Presenter can easily implement the GetController method, since I had come up with a base Controller class. But the LoadModel method was different, because there was no base Model class. Rather, sometimes the Model would be an Entity, and sometimes the Model would be an EntityCollection, and sometimes the model might be something else. I put the two entity classes side by side in Visual Studio, and looked that their respective declarations:

public abstract class Entity
{
}

public abstract class EntityCollection : CollectionBase
{
}

The Entity class had no base class; it is a brand new class. The EntityCollection, on the other hand, derives from the existing CollectionBase class. What do these two classes share in common, such that the Presenter has a known type to work with? This had me scratching my head and ultimately putting pencil to paper. I thought about introducing a new Model class that contained one Entity and one EntityCollection, and the consuming Presenter would have to test which item was not null. I then thought about introducing an IModel interface, and force both the Entity and the EntityCollection classes to implement that interface. I then come up with other ideas too, but got so confused that I ended up taking a long ride on my motorbike:

The answer to this problem is embarrassingly obvious. However, I was so focused on Charlie that I had completely forgotten the reality that I was developing against the .NET Framework Class Library. While I was out riding the obvious answer became, well, obvious. I had clear forgotten that every single object in a .NET application derives from the base System.Object class. While it appeared that the Entity class and the EntityCollection class shared nothing in common, that was untrue. Both derive from the base Object class.

public abstract class Presenter
{
    public abstract Object LoadModel(Int32 id);
    public abstract View GetView(ViewMode viewMode);
    public abstract Controller GetController(ViewMode viewMode);
}

Having written the above description, I am very tempted not to publish this weblog entry. The fact that I had forgotten the base System.Object class is truly embarrassing. Who would ever trust a .NET developer that forgets this fundamental class? So, while I am deeply embarrassed by this gaffe, I did promise that this blog would be a true account of “the trials and errors of a self-taught developer creating his first web application.” So there you have it: the dumbest mistake I have made so far, and an example of this duck paddling madly beneath the surface.

by Alister Jones | Next up: A Dummy’s Approach to Agile Development →

Lessons Learned

2006-03-02T11:33:00.000-08:00

Over the last few weeks I have been bringing the story of Charlie up to date. The last few weblog entries concerned some fundamental pieces of Charlie’s interface system. With those basic interface pieces in place, Charlie can now serve simple, secure, localized, and configurable webpages. If you refer back to the software specifications, you will see that this means Charlie is adhering to its specifications. And at this point, the story of Charlie is now up to date.

Right now, all configuration is carried out directly in SQL Enterprise Manager. There is no administration interface that allows a website owner to configure his or her own website. That’s okay, because it means I have been concentrating on the function, and not the form, of the application. But the time has come to add some flesh to the skeleton of Charlie. (While that is a common and helpful metaphor, it really is quite gruesome when you think about it. Urgh!)

My last weblog entry was a failed attempted to suggest that this next stage in the development of Charlie will consist of three stages. First, I need to design the administration interface. Second, I need to hand-craft the design of the administration interface. Third, I will need to implement that hand-crafted design.

Before I move on to the design of the administration interface, I thought I would pause to describe some of the lessons I have learned in getting Charlie to its current point. That is what I will do in the next few weblog entries.

by Alister Jones | Next up: Lesson: Don’t Forget .NET →

The Craft of Web Design

2006-03-02T10:24:00.000-08:00

I would like to talk a little about the craft of web design. Well, I’d actually like to talk a lot about the craft of web design, but I will try to get my point across and then stop.

To start with, let’s distinguish craft from design. To paraphrase [name removed—see end of entry], we can say that design is the process by which we make tradeoffs to arrive at a solution. Poor design comes from making poor choices in the tradeoffs. Good design comes from making good choices in the tradeoffs.

To add visual interest to this weblog entry, let’s take the example of creating an advertisement for a new Marilyn Monroe movie, the title of which is An Office Affair. The design aspect of creating the advertisement involves making tradeoffs. The ad should be big, so that it can be seen. The ad needs to be small, to minimise its placement cost. The ad should provide lots of text, so the customer can determine if he or she wants to see it. The ad should provide little text, so the customer is intrigued. The ad should feature a large photo of Marilyn Monroe, as she is the bankable star. The ad should feature a small photo of Marilyn, to leave room for the text. These are all tradeoffs that must be made.

Let’s look at what must surely be a poor design, because it has made poor tradeoffs.

Let’s now look at what we might consider a good design. For the purposes of this weblog entry, it does not matter whether it is in fact a good design. Let’s just presume that it has made good choices in the tradeoffs, and is therefore good design.

Remember now, it is not important whether this actually a good design. We are just presuming that this is a good design, which is to presume that it makes the right choices regarding the size of the advertisement, the size of the photograph, and the amount of text.

While the above is good design, it is still not a good advertisement. While the right tradeoffs have been made, the result looks crappy. Why?

A fairly common sales pitch by advertisers is some variation of the statement, “First we make it right. Then we make it wonderful.”

Craft is what will move the above “right design” to the final “wonderful design.” Let’s have a go at crafting this advertisement without changing any part of its design (we do not change the size or placement of the photograph or the text). What I hope you will notice is that craft can make a significant difference to the advertisement, without changing its fundamental design.

First, let’s change the photography from being blocked to being etched.

Let’s now make the background black.

The photo now looks quite flat against the black ground. Let’s give it some oomph.

Next up, let’s change the typeface to one that is more classical.

To make the typography more graceful, let’s introduce the required ligatures. If you are not sure what a ligature is (and you did not click on the preceding link), the following image shows the changes.

With the support of ligatures, we arrive at the following.

Finally, let’s kern the letters.

Compare this final advertisement to the original.

Notice that the design (the size and placement of the photograph, and the size and amount and placement of text) has not changed at all, but the craft has changed the …

You know what? This is a terrible example. It is so bad that I have removed the name of person I was attempting to paraphrase, as the above example is a disservice to his description of what design is, and what it is not. However, this weblog entry taken me so long to put together that I don’t have the heart to trash it.

Can we just say that I put an enormous amount of value in the craft of web design? No detail is too small to be important. Fortunately, that vaguely ties in to the following weblog entry.

by Alister Jones | Next up: Lessons Learned →

The Presenter Object

2006-03-02T09:25:00.000-08:00

In the last weblog entry, I said that I would describe how an entity such as an Article gets onto the page. I spent time describing the business objects involved in the process, but I did not really describe how the Container object gets and presents an entity. I’ll do that in this weblog entry.

In the first instance of Charlie, I created a class named an Applet. This class inherited from ASP.NET’s UserControl class, and exposed an EntityId property. Charlie’s equivalent of the PageEngine would “ask” the Container to give it an Applet. The Container object then set the Applet’s EntityId property, thereby telling the Applet which entity it is to display. For example, an ArticleApplet would be a UserControl that includes a Title Label, a Summary Label, and a Content Label. The ArticleApplet looks at its EntityId property, which was set by its Container, to learn which Article entity to load up. After it has obtained the corresponding Article, it uses the article’s Title, Summary, and Content properties to populate the corresponding Label controls.

Since an Applet inherited from the UserControl class, the PageEngine simply dropped it onto the page, allowing the ASP.NET process to do its thing. This Applet approach proved to be a very simple way of displaying entities such as an Article. If it was simple, and if it worked, why did I change it?

For a simple item of content such as an Article, the above approach was adequate. However, for more complex content, such as a collection of blog entries which exposes Atom or RSS feeds, and accepts and presents comments from visitors, the approach was too simple. The UserControl would become very complex, accepting far too many responsibilities for an interface element. What was missing was the Controller part of the Model-View-Controller pattern.

I still liked the simplicity of the Applet approach, but I wanted to introduce the flexibility provided by the MVC pattern.

I started by renaming the Applet class as the View class. This was a simple change that formalised the part played by the UserControl-based object in the MVC pattern. Whereas the earlier ArticleApplet object had many responsibilities, including presenting an article and controlling actions such as a visitor adding a comment to the article, the new ArticleView object was responsible only for presenting the article.

The next step was to introduce the Controller object. The responsibility of the Controller object is act as the middle-man between the View (in the Presentation Layer) and the Managers (in the Business Layer). Without the Controller, the View needs to talk directly to any number of different Managers (to present an Article might require the View to talk to an ArticleManager, a CommentManager, and an AuthorManager). With the Controller in place, the View is allowed to stick to its two main responsibilities: to present the Model to the user, and to interpret actions from the user.

Is this a “true” implementation of the Model-View-Controller pattern? Hell, no; it’s not even close. However, this simplified implementation does what I wanted: the UserControl-based View object is freed from knowing about which Managers may need to become involved. All the View knows about is the Model it is to present, and the Controller it is to tell about any user actions.

Only two questions remained to be answered. First, which object would be responsible for creating this trinity of a Model object, a View object, and a Controller object? Second, how would these three objects be “packaged”?

Now, which object would be responsible for creating the package of a Model, View, and Controller? The first option would be to make the Container responsible, since this is the object that the PageEngine talks to when it comes time to place the UserControl-based View objects onto the page. However, this would give the Container a whole different set of responsibilities, which adversely affects its cohesion. To avoid complicating the Container object, I decided to introduce a new little factory class, and make the factory responsible for packaging up a Model, a View, and a Controller. My initial instinct was to call this class an MvcFactory. But, not only was this clumsy, it would become inaccurate if later developments to Charlie meant that the MVC pattern was replaced by some other approach. So, I scribbed down some alternative names and settled upon calling this class a Presenter. To my mind, having an ArticlePresenter and a WeblogPresenter would nicely balance the existing ArticleManager and WeblogManager.

The other question was how the Model, the View, and the Controller would be “packaged” and returned to the PageEngine. The Presenter could return the three objects separately, and leave it to the PageEngine to tie them together. That was a viable option, but it was more complex than the existing solution where the Container returned a single UserControl-based object that the PageEngine could drop on the page. So, yet again following the guideline that simple is better than complicated, I decided that the UserControl-based View object would comprise its Model and its Controller, so that the PageEngine only ever sees an incoming View, and never needs to know that the View represents an MVC “package”. Even better, it supports the possibility that sometimes the View will be a dirt-simple UserControl, with no Model and no Controller. So not only was this approach simpler, it was more flexible, too.

So there you have the description of how a little more of Charlie’s architecture has been designed:

by Alister Jones | Next up: The Craft of Web Design →

Getting Entities onto the Page

2006-03-02T08:27:00.000-08:00

In my last weblog entry, I said that every item of content on a Charlie webpage is based on the Entity System. But how do these business objects get onto the page? For example, how does an Article business object get the chance to display its Title, Summary, and Content properties on the webpage for the visitor to read? The solution to this challenge involved two stages. I will talk about the second stage in the next weblog entry.

The first stage of getting a business object onto the page involved copying the approach taken by Cuyahoga. (If Charlie ever helps me to earn an income, Martijn Boland is going to get a few bucks.)

The very first version of Cuyahoga includes a Node business object. If you picture a sitemap, each page in the sitemap will be represented by a Node business object. Why do we need a Node business object to represent a webpage, when typically we already have an .aspx page? Well, like most website frameworks, Cuyahoga does not include an .aspx file for each and every webpage in a website. Rather, Cuyahoga includes a PageEngine class that dynamically creates the webpages that will be seen by a visitor. Given that there is no actual .aspx page, we need an object that represents that page. That object is the Node object.

Each Node business object will include a collection of Section business objects. As its name suggests, a Section represents one section of the webpage that is represented by the Node.

Each Section object comprises a Module business object. Concrete examples would be an ArticleModule and a BlogModule. The Module class includes a property that “points to” the .ascx User Control needed to display the Article or BlogEntry or other object.

When a visitor requests a page, Cuyahoga’s PageEngine object does the following. First, it gets the appropriate Node object. Then, it loops though each Section object in that Node. For each Section, the PageEngine will get the Module object that belongs to the Section. The Module will be inspected to determine which .ascx User Control it needs added to the page. The PageEngine will load up an instance of the User Control, and will add it to whatever PlaceHolder server control is defined by the Section. The PageEngine will then “give” the User Control its parent Section. This allows the User Control to inspect the Section object and its Module object, in order to know which Article or BlogEntry or other object it is to display.

This is how it worked in version 0.1 of Cuyahoga. A cursory glance at later versions of Cuyahoga suggests that this process has been polished. Whether it has or not is unimportant to dear old Charlie, since the simplicity of this process was just fine for Charlie in the beginning. (I’ll explain my own polishing steps in the next weblog entry.)

I did make some superficial changes to this process, by way of renaming a few of these business objects. If you look back at the UML diagram I originally created, you can follow my thought process.

The second-top object in the hierarchy is a Domain object, which represents an internet domain. The Domain object will include the true domain, such as “example.com”, and may include a collection of aliases, such as “localhost/example”. The Domain object may include other relevant details such as its WHOIS information.

The top object is the Owner, which is of course the owner of that internet domain. The Owner may be a person or an association or a company.

The Resource object represents an internet resource, as per the definition of a URI (Uniform Resource Identifier). The Resource will include little more than an address such as “/toys/elmo”.

A resource, or web address, points to a Document object. Cuyahoga uses a Node object, but Charlie and I prefer to call this a Document object. After all, each file returned from a domain includes a description of its document-type (such as “text/html” or “image/jpeg”). So that was a superficial change I made from the business objects used in Cuyahoga.

Each Document will use a Template that provides the structure of the document. If I were making best use of ASP.NET version 2.0, this would be implemented by way of a MasterPage. However, I have elected to follow Cuyahoga’s example and use a simple .ascx User Control.

A Template will be divided into Sections, such as “header”, “content”, and “footer”. This is not the same meaning as Section in Cuyahoga, so this too represents a variation. To my mind, a section of a webpage document is a visible “area” of that webpage, and into that section can go many different things (into the “header” section can go the logo, the search box, and the navigation bar). In Cuyahoga, each Section includes only one Module, so I found this term caused me a little confusion. So in Charlie, a Section means one area of the Template that structures a Document, into which any number of things can be placed.

Into each Section a collection of Container objects will be added. A Container contains one, and only one, item of Content. (So Charlie’s Container object has the same purpose as the Section object in Cuyahoga.)

A Content object displays an item of content, such as an Article or a BlogEntry.

An Asset is the item of content that is displayed by a Content object, such as the Article or the BlogEntry. If you find the distinction between a Content object and an Asset object a little unclear, you’re not alone. As I was developing Charlie, I found no use for the Content object, so I removed it from Charlie’s code. The fact that I had included a useless class in my UML diagram shows that the diagram is faulty. Still, this weblog entry is an honest account of Charlie’s development, errors and all.

So there you have a long description of how Charlie gets a business entity, such as an Article or a BlogEntry, onto the webpage that is returned to the visitor. Aren’t you glad you asked? Oh, you didn’t ask? Hell, then I’m sorry to have provided such a boring description. If I provide a sexy picture, will you forgive me? I thought so.

You don’t find saucy photos like that everyday on the internet, do you?

by Alister Jones | Next up: The Presenter Object →