Standing on the Shoulders of Giants

Tiny-Type Language Support

2011-08-16T19:14:00.000-07:00

Tiny-Type Language Support

Tiny-Types are types in object-oriented programming that contain very little data, but still have the benefits of objects. The typical example is currency. Tiny-types are underused which in part is due to poor language support. Although some of these shortcomings can be solved by code generation, it would be beneficial if such friction weren't necessary (although Scala case classes take care of some of these concerns). There are several features that are missing in mainstream languages that would make using tiny-types easier.

Built-in hash code

Many IDEs provide support to generate hash codes automatically. However, this does not stop them from getting out of sync, as they are not automatically regenerated when the types or number of fields change. This can be provided in an IDE, but it would be easier to just provide this mechanism in the language. Say as an alternative to a class, one could use the keyword value that would automatically provide the hash code for you given the fields in the type.

Built-in equality

Equality is another method that can be generated by the IDE but can become stale when the types or number of fields change. This can be implemented by the same mechanism as with hash codes. Additionally, there should only be one method/operator that does equality. In Java there is .equals() or operator== depending on whether or not the type is a class or primitive. In C#, you have to implement both if you want proper functionality as operators are overloadable. The in-fix notation, which is used by operator==, is more habituated due to the way most of us learned math. Therefore, the method .equals() should be eliminated, leaving one place for both primitives (values) and classes to define equality.

Built-in JSON/binary serialization

Tiny-types are convenient as messages where message passing between processes or threads are used. While serialization is not necessary between threads, it is essential for communication between processes. Both JSON and some form of standardized language-independent binary serialization (perhaps Message Pack, Thrift, or gobs (but language-independent?) should be generated for value types by default.

toStr() defaults to JSON

String methods are another facility that can be generated by an IDE or by code generation, but again this should be unnecessary for value types. The default for toStr() method should produce JSON. As with serialization, equality, and hash codes, toStr() should be overrideable.

Immutable

Object level immutability is sorely lacking from many mainstream object-oriented languages, with the exception of C++. Value types should be immutable, allowing for guarantees with intra-process message passing. Immutability is also useful for hashing.

Operator overloading

Operator overloading is very important for tiny-types, particularly for mathematical operations. Anyone who has ever used the terrible syntax required by BigDecimals in Java can attest to how nice it would be to have operator overloading. One of my main complaints about Go is the lack of operator overloading. I understand why it was left out, but I believe the pros outweigh the cons. C# strikes a nice balance here by allowing the overloading of mathematical operators but not assignment or arbitrary operators.

Tiny-type support is lacking in most modern mainstream object-oriented languages. This is unfortunate as it takes away from the full benefits of object-orientation and would be easy to remedy by language designers.

Refactoring: Extract setup and helper methods into test scenario

2011-05-23T17:52:00.000-07:00

In my last post, I discussed being able to implement the entire application-under-test in one line. This technique is useful at more granular levels as well. In fact, it offers a nice alternative to the idea that tests should be wet (not-DRY). Object-orientation turns out to be a nice solution to duplication in tests.

Refactor this:

[TestFixture]
public class MyObjectTests
{
    // field declarations omitted for brevity

    [SetUp]
    public void Setup()
    {
        _dataStore = new StubDataStore();
        _networkConnection = new StubNetworkConnection();
        _collaborator = new Collaborator();
        _serializer = new Serializer();
    }

    private MyObject CreateSut()
    {
        return new MyObject(_dataStore, _networkConnection, _collaborator, _serializer);
    }

    private static void CustomAssertEquals(IDictionary<string, string> expected,
        IDictionary<string, string> actual)
    {
        // custom assertion
    }

    [Test]
    public void ShouldValidateAdditionalNewFeature()
    {
        var sut = CreateSut();
        sut.LoadDataStore();
        sut.BroadcastStateOnNetwork();

        CustomAssertEquals(_dataStore.Data, _networkConnection.Data);
    }
}

Into this:

public class MyObjectScenario
{
    // field declarations omitted for brevity

    public MyObjectScenario()
    {
        _dataStore = new StubDataStore();
        _networkConnection = new StubNetworkConnection();
        _collaborator = new Collaborator();
        _serializer = new Serializer();
        _sut = new MyObject(_dataStore, _networkConnection, _collaborator, _serializer);
    }

    public void AssertNetworkMatchesDataStore()
    {
        // custom assertion
    }

    public void LoadDataStore()
    {
        _sut.LoadDataStore();
    }

    public void BroadcastStateOnNetwork()
    {
        _sut.BroadcastStateOnNetwork();
    }
}

[TestFixture]
public class MyObjectTests
{
    [Test]
    public void DataBroadcastOnNetworkShouldMatchDataStore()
    {
        var sut = new MyObjectScenario();
        sut.LoadDataStore();
        sut.BroadcastStateOnNetwork();
        sut.AssertNetworkMatchesDataStore();
    }
}

Now each test case in the TestFixture is succinct, independent (there are no instance variables in the TestFixture), and is very readable. Additionally, the test cases are DRY and the assertions print out very meaningful error messages which are reusable across TestFixtures. Notice that I am not using a mocking framework for the out-of-process dependencies (network, data store) as this allows me to DRY up the behavior I would otherwise have to redundantly put in each setup or test case. Lastly, these scenarios can be chained up to create the whole application-under-test in my previous post, with little duplication.

It is beneficial to keep your tests DRY. Tests are code too and reuse is just as helpful in tests as it is in production code.

Can you instantiate your application-under-test in one line?

2011-03-26T16:24:00.000-07:00

Can you instantiate your application-under-test in one line? As in your entire application, with all externally facing interactions (threads, GUI, databases, network, etc.) faked out?

Like this:

[TestFixture]
public class MyApplicationFunctionalTests
{
    [Test]
    public void ShouldValidateAdditionalNewFeature()
    {
        var sut = new MyApplication();
        sut.DataArrivesOnNetwork("message from another user");
        sut.AssertGuiDisplays("message from another user");
    }
}

The ability to instantiate your application-under-test in one line enables you to immediately and easily write a test for any new feature or bug fix, and see the effects on the entire application. The power of this technique is difficult to overstate. The true benefits of TDD are much more evident using functional testing with real classes at the application level than at the unit test level using a mocking framework.

I picked up this technique from Growing Object-Oriented Software, Guided By Tests. While the authors and I disagree about the effectiveness of mocking frameworks (they are the authors of JMock2), we certainly agree on the use of functional testing at the application level (we also agree on DI containers being detrimental). I tried the AUT-in-one-line technique with a recent greenfield-ish project and the results were very positive. All projects I start going forward will definitely use this pattern.

Chrome Wishlist

2010-12-03T18:47:00.000-08:00

I am unabashedly a huge fan of Chrome. Minimalism is emphasized throughout its design and it is very fast. However, there are a few small outstanding issues that for some reason have been overlooked.

Wasted space at top of window

The following pictures on the top are what Chrome currently looks like. The pictures on the bottom are what Chrome would look like without the extra wasted space at the top (also what it looks like maximized on Windows/Linux). Why is it there? To move the window? How often does that happen? Is it worth all that space? There still would be plenty of space around the min/max/close buttons to perform such a rare operation.

Windows 7

Ubuntu

Mac OS X

Inconsistent keyboard shortcuts between platforms

On the Mac, the Bookmark Manager shortcut is Alt+Command+B. On Windows/Linux, it is completely missing. On Windows/Linux, the Task Manager shortcut is Shift+Esc, on Mac, it is completely missing. Why the inconsistency?

Remove bookmark bar

The bookmark bar is not useful. If there are a few bookmarks that are often used then the sites will readily come up in the omnibar. In keeping with Chrome's minimalism, it should be removed.

Sync search engines

Many pieces of data are synchronized via Google Chrome. One thing that is definitely missing is synchronization of search engines.

Add bookmark search engine in omnibar

The search engine support in the omnibar is very useful. It would be more efficient if I could search just my bookmarks via the same mechanism, instead of having to open up the bookmark manager page.

Retlang Documentation Improved

2010-09-29T08:57:00.000-07:00

Now that Retlang has reached 1.0 status, the next task was to clean up the documentation. Several areas have been improved.

Obsolete documentation has been removed. Majority of “getting started” information is now directly on the front page.
Video/presentation of graphical guide to Retlang is posted and linked on the front page.
Classes and interfaces have been renamed in order to more easily grok the library.
Google Group has been cleaned up. Spammers have been removed and all posts are now moderated before appearing.

Please let me know (via the user group or this blog post) of any other improvements that could be made.

Next Project

2010-07-09T17:57:00.001-07:00

I have been working on one large project for a few years now, and while we have done a pretty good job in keeping the code as clean as possible, there are still some endemic characteristics I would like to change if I were to start over.

No DI Framework
We started with Spring.NET, but then ran into the usual XML problems (renames, namespace changes, ugly factory syntax, ugly generics syntax). So then we moved to Spring-Recoil, a programmatic abstraction over Spring.NET. Unfortunately, the following are still painful:

Missing constructor arguments are run-time errors.
Conditionally loading containers (DI frameworks are declarative).
Custom object life cycles.
Substitution of individual classes within a container (useful for creating integration test harnesses).

Yes there are other DI frameworks out there, but they all still suffer from the above. While dependency injection is invaluable, using a DI framework is more trouble than it’s worth. As a coworker recently said: “Spring is training wheels for learning dependency injection.”

Enforce no more than 4 arguments per method/constructor
I picked this one up from Clean Code by Uncle Bob. This is an easy, concrete way to enforce small class/method sizes and Single Responsibility. Vastly superior than line count. A reflective automated test will keep this enforced.

Virtually no mocks
There is no easier way to make a brittle test whose functionality has no bearing on the functionality of the system than mocks. Mocks are for testing how exactly one object interacts with another. This not useful information. I want to know the result. If there is interaction with another object, and I want to isolate the system-under-test, then I will use a stub, hand-rolled if I need a little functionality. However, I usually prefer to use as many real classes as possible, as I am more closely matching the functionality of the real system (in which case "unit" testing is not the best label).

Enforce a test class for virtually every class
I have tried to religiously practice TDD, but still haven’t gotten there. For me TDD is more helpful for ensuring correctness than for design as I rarely come back to a class I have trouble testing. An automated test to reflect through the code and enforce the existence of a class named $class.name$Tests should do the trick, as I am pretty good at keeping the build green. For classes where this is unnecessary, I will add a white list.

Web UI
Google Chrome has recently advanced to the point of being able to provide parity with fat client applications. SVG, Canvas, WebSockets, CSS3, etc. allow for rich user interaction. Coupled with the vast performance gains in the WebKit rendering engine and the V8 Javascript engine, Chrome is quite fast as well. Now that it matches fat clients in terms of speed and richness (in most cases, hardware acceleration is still lacking), the advantages of cross platform and central deployment that thick clients lack make web apps the clear winner. Note this isn’t appropriate for all applications, but is for the domain in which I work.

Retlang 1.0.1 Released

2010-06-23T21:18:00.000-07:00

This is the final scheduled release. No known bugs or further optimizations exist and the library has been used extensively in production.

Targets .NET Framework 4.0 (generic bug fixed).
Lists are recycled between executors and fibers, instead of creating new arrays on each execution.
StubFiber now has ExecutePendingImmediately defaulted to false. The new default more closely resembles how real fibers execute. Also it matches Jetlang.
StubFiber now executes pending actions in FIFO (as opposed to LIFO) order when actions enqueue other actions. In previous releases this was an issue only when ExecutePendingImmediately was set to true.
Additional constructors for DispatcherFiber and ThreadFiber.

Get it here.

Missteps in the Design of the C# Language

2010-02-20T10:16:00.001-08:00

C# as a language has many benefits over Java. Some of the more noticeable include a proper implementation of generics, anonymous delegates and lambdas, and niceties such as the var keyword, IDisposable, and LINQ expressions. However, there are several parts of the C# language that are poor, and would have been (are) easy to fix.

Enumerations

Why can I not add methods to enumerations as I can in Java? The only time I use C# enumerations anymore is for DTOs. Otherwise, I know from experience that I am going to end up with a switch statement on the enumeration when I should be using polymorphism. See below:

enum Fruit { Apple, Orange }

vs.

class abstract Fruit
{
    public static readonly Fruit Apple = new Apple(1);
    public static readonly Fruit Orange = new Orange(2);

    protected Fruit(int id)
    {
        _id = id;
    }

    public int Id
    {
        get { return _id; }
    }

    public abstract virtual void Peel();

    private class Apple : Fruit
    {
        public Apple(int id) : base(id)
        {}

        public override void Peel()
        {
            // omitted for brevity
        }
    }
    // rest omitted for brevity
}

Events

Events are probably the worst offender. Why is there an event keyword? Why not just use the already existing multicast delegate? Is not having to encapsulate the field really worth the confusion it causes? Public events appear to be able to be referenced by outside classes. This is however not the case, some methods can only be accessed by the owner class. For even more fun try passing an event to a method. Also why not use the null object pattern with events instead of forcing me to do this every time:

public event EventHandler ButtonPushed = delegate {};

An argument against this is that invoking a null event should result in a null reference because it was never allocated. Well then why can I append to it? I can't add an element to a null list reference, then why can I append an EventHandler to a null event? The most consistent solution would be to require allocation before appending.

Structs

It is very useful to be able to allocate data on the stack. Unfortunately if you have any references to classes, then your struct gets boxed and goes on the heap. Structs should have been named value, so as not to confuse them with the unrelated C++ structs and to be more intention revealing. Also the compiler should prevent you from referencing any classes, as to preserve the benefits of stack allocation.

// this shouldn't compile
struct Currency
{
    public int Dollars;
    public int Cents;
    public string Symbol;
}

// this should compile
value Currency
{
    public int Dollars;
    public int Cents;
}

Nullable<T>

A coworker introduced me to this one inspired by his learning of Haskell and the Maybe monad. References to classes should be NOT nullable by default. References to classes that can be null should be marked with a ?. Imagine how useful it would be to know which variables could be null and the convention was enforced by the compiler?

public static void MethodToCall(RequiredArg reqArg, OptionalArg? optional);

// should not compile
MethodToCall(null, null);

// should compile
MethodToCall(new RequiredArg(), null);

While C# as a a language definitely has its benefits over Java, there are still some warts that could definitely be fixed.

StubFiber: How to Deterministically Test Concurrent Code

2010-01-16T11:44:00.001-08:00

Writing correct concurrent code is considered by many to be one of the more difficult areas of computer science. Verifying the correctness of concurrent code is deemed even more difficult. Ask most developers how to verify the correctness of concurrent code deterministically, and you will be often met with a blank stare. Even books such as Clean Code, an otherwise worthwhile read, is incredibly disappointing when it comes to testing concurrent code. An entire chapter and appendix is dedicated to the topic of clean concurrent code. The recommendation comes down to abstracting away threading (but with no examples of how to do this) and using third party tools to alter the thread scheduler to introduce artificial pauses in the hopes that they will expose bugs.

It turns out that writing correct concurrent code and testing it deterministically is actually not that difficult. As the saying goes, if it hurts, you are doing it wrong. The first step is architectural. Instead of using shared-state concurrency with all the perils of nested locking, use message based concurrency. Keep unparallalizable mutable state in a single thread and then communicate with other such threads via immutable messages. Two libraries written by Mike Rettig, called Retlang (for C#) and Jetlang (for Java) greatly assist with message based concurrency by introducing the Fiber and Channel abstractions. Fibers are abstractions for threads and Channels are abstractions for the conduit through which Fibers send and receive messages from other Fibers. Since a Fiber is inherently synchronous with itself, it processes all of its messages in order, eliminating the concern of one message handler interrupting another.

By relying on the Fiber abstraction instead of threads themselves, ThreadFibers or PoolFibers can be substituted with StubFibers to obtain synchronous deterministic tests. Any race conditions between threads can be made explicit in a test and thus correct execution can be validated for all interleaves of the thread schedule. Below is an example in Retlang (C#):


public class Runner()
{
    private readonly Channel<CalculationResult> channel = new Channel<CalculationResult>();
    private readonly Channel<object> requestChannel = new Channel<object>();

    private readonly StubFiber fiberOne = new StubFiber { ExecutePendingImmediately = false; };
    private readonly CalculatorOne calculatorOne;

    private readonly StubFiber fiberTwo = new StubFiber { ExecutePendingImmediately = false; };
    private readonly CalculatorTwo calculatorTwo;

    private readonly ResultAggregator resultAggregator;

    public Runner()
    {
        calculatorOne = new CalculatorOne(fiberOne, channel, requestChannel);
        calculatorTwo = new CalculatorTwo(fiberTwo, channel, requestChannel);
        resultAggregator = new ResultAggregator(new StubFiber(), channel, requestChannel);
    }

    public void ExecuteAllPendingOnFiberOne()
    {
        fiberOne.ExecuteAllPending();
    }

    public void ExecuteAllPendingOnFiberTwo()
    {
        fiberTwo.ExecuteAllPending();
    }

    public void StartCalculations()
    {
        resultAggregator.StartCalculations();
    }

    public CombinedResult CombinedResult
    {
        get { return resultAggregator.Result; }
    }
}

[Test]
public void FiberExecutionOrderShouldNotChangeResult()
{
    var runner = new Runner();

    runner.ExecuteAllPendingOnFiberOne();
    runner.ExecuteAllPendingOnFiberTwo();

    Assert.AreEqual(CombinedResult.Expected, runner.CombinedResult); 
}

[Test]
public void SecondFiberExecutionOrderShouldNotChangeResult()
{
    var runner = new Runner();

    runner.ExecuteAllPendingOnFiberTwo();
    runner.ExecuteAllPendingOnFiberOne();

    Assert.AreEqual(CombinedResult.Expected, runner.CombinedResult); 
}

Message based concurrency is vastly superior to shared state concurrency. Not only is the resulting design significantly easier to understand, it also has the nice side effect of being able to be tested deterministically.

Circuit Breakers

2010-01-09T13:13:00.001-08:00

It is extremely difficult to write bug-free software. Even if you managed to acquire the most talented developers in the world, gave them plenty of time to release, allowed them to use the best suited methodologies for the project, and had well-specified requirements, I'm confident the resulting software would still have bugs. There are many reasons behind this unfortunate truth. According to Fred Brooks in The Mythical Man Month, software construction is probably one of the most complex endeavors yet undertaken by human beings.

Many techniques are helpful to reduce the amount of bugs in a codebase. Writing clean readable code is probably the easiest. Several levels of testing (unit, integration, functional, user, load, performance, etc.) are the most obvious. In Facts and Fallacies of Software Engineering, Robert Glass shows that code inspection (which includes pair programming) when done frequently and in small groups is also an effective way to reduce bugs. Unfortunately, all of these techniques will only reduce the number of bugs, not guarantee the elimination of all of them. As developers we have to accept that we will ship software with bugs.

The question is what can we do as developers to reduce the impact of bugs as much as possible given that we know they will occur in production. Michael Nygard, in his book Release It!, introduces what he calls the Circuit Breaker pattern. The software pattern matches very closely to the real world pattern. No circuit in your house should draw more current than it is rated to handle. Of course at some point, somewhere, someone will plug in enough power hungry devices to overload a circuit. Before circuit breakers, this would cause a fire. With circuit breakers, the circuit is broken, and the house remains safe.

This technique involves more than a simple try-catch block. It involves keeping timeouts for interactions with external systems. It involves limiting the total number of times an operation will be retried before alerting the user to the problem. It involves monitoring resources such as disk space or memory and terminating non-essential actions until the resources are freed. It involves limiting the number of rows that can return from a database query.

While every reasonable effort should be made to eliminate bugs, as a professional developer you have to admit that bugs are not 'ifs' but 'whens'. Circuit Breakers are an excellent defense against the intractable reality of software bugs.

The Benefits of Unstructured Data

2009-12-09T16:11:00.001-08:00

In the book The Pragmatic Programmer, an entire section is dedicated to "The Power of Plain Text". The section lists integration (the UNIX tool chain being the canonical example), insurance against obsolescence, and easier testing as motivations for using plain text. There is another significant advantage to plain text that is often overlooked: it lacks rigorous structure.

MongoDB

There has been a trend recently against relational databases that has been coined NoSQL. Not to say that relational databases are always a bad idea, just that they are not the best tool for every job. Personally, I have all too often encountered the "I need to persist data so I must use a relational database" mentality. Avoiding RDBMS allows for avoiding schemas. Database schemas are a pain because they have to stay synchronized with all of the applications that access it. Typically this also involves going through a DBA group, which adds a hop that impedes rapid and frequent deployment. Ruby on Rails has a facility called migrations to help deal with such pain points, but avoiding schemas removes this problem altogether. Furthermore Rails maintains referential integrity at the application layer, making an RDBMS largely overkill. Although my experience with non-relational data stores is far from comprehensive, my favorite so far is MongoDB.

The first thing to notice about MongoDB is the API. The API is very simple and has little if any accidental complexity. There are named collections and you can store dictionaries (or nested dictionaries) in those collections. Each dictionary within a collection can have a different number and types of key-value pairs, and they can be nested. Simple.

The benefit of not having schemas is that the application can be upgraded before the data store is ready. Just deploy the application upgrade into production and start storing the new data. Separate applications can in most cases be upgraded independently. If previously stored data needs to be normalized, this can happen separately, before or after the application upgrade occurs.

In addition to avoiding schemas, an unstructured approach yields two other benefits. Want access to the data independent of MongoDB? The data is all stored in binary JSON, and therefore is accessible from other languages and applications that have no knowledge of MongoDB. Another benefit of MongoDB made possible by its non-relational nature is support for auto-sharding and MapReduce. These features make MongoDB scale significantly better than RDBMS solutions. MongoDB nicely fills niches on both sides of RDBMS in the size-of-data spectrum.

Wikis

Wikis are an excellent way to share content and communicate. Wikis are superior to other solutions, such as SharePoint, because they are akin to a version controlled notebook that everyone can view/edit from their browser. Notebooks impose no structure, nor require a separate context to view or edit (such as Word). Many wikis have macros for tables and other formatting but these are all optional. The wiki will not complain if you leave something out. Wikis can make an excellent replacement for documentation, forums, task lists, release notes, etc. One of the more recent items I have replaced with a wiki is Mingle, a software project management tool. Having used several project management tools, I find a wiki to be a better choice due to their lack of structure. The model imposed by project management software never quite fits or is too heavy handed, forcing me to either make my process match the tool, or spend more time than it is worth to coerce the tool to fit my process. Lastly, searching is significantly easier with a wiki because the data is all text. Searching in SharePoint is frustrating in comparison.

The "Power of Plain Text" is understated. Rigorous structure is not always a good thing when it comes to data, and less stringent solutions should be considered first, as they are often more flexible and require less developer overhead.

Encapsulate Collections and Primitives

2009-10-24T11:13:00.001-07:00

I recently came across an article entitled Object Calisthenics by Jeff Bay. The purpose of the exercise is to show the effects of following object-orientation rigorously on a small greenfield project. After the calisthenics are practiced as rules, they are to be relaxed and used as guidelines in production code. Two of these exercises are of particular interest.

Rule 3: Wrap all primitives and strings

As referenced above, the goal of rule 3 is not to preclude the use of naked primitives and strings altogether, but to learn when it is appropriate to encapsulate them. Interestingly, the language of choice affects the overhead of encapsulating primitives and strings.

In Java a wrapped primitive must be an object, thereby adding work for the garbage collector. Secondly, Java does not allow operator overloading thus adding a lot of ceremony to otherwise simple mathematical operations. Anyone who has suffered through the use of BigDecimals has experienced this pain.

In C#, primitives can be wrapped as structs and therefore be allocated on the stack, making GC performance less of a concern. Also, C# allows for operator overloading of arithmetic operators, eliminating the ugliness of using BigDecimal. C# also supports implicit casting, making conversion between primitives and a wrapped class succinct, though I have to admit I never use implicit casting because I am always worried it will cast unexpectedly.

Moving past language support, when are the appropriate times to encapsulate strings and primitives? Type safety in statically typed languages is the first that comes to mind. Both for mathematical operations and argument passing. If a method takes two different doubles, it can be refactored so it takes two separate types. This helps eliminate the mistake of transposing the arguments. Another advantage of type safety is to restrict mathematical operations to only a subset of types. F# does a good job of this with its units library. Below is a trivial example using C#.

// Mechanics.cs
public Mass Calculate(Force f, Acceleration a)
{
   return f/a;
}

// MechanicsTests.cs
private readonly Force f = new Force(12);
private readonly Acceleration a = new Acceleration(4);
private readonly Mass m = new Mass(3);

[Test]
public void ForceShouldBeMassTimesAcceleration()
{
    Assert.AreEqual(m, new Mechanics().Calculate(f, a));
}

[Test]
public void WillNotCompileButWouldUsingPrimitives()
{
    Assert.AreEqual(a, new Mechanics().Calculate(f, m));
}

The method Calculate could have a better name, but type safety makes it largely redundant. Type safety also makes it safer than a more intention revealing method name as the compiler does not enforce the intent of method names.

Another reason to encapsulate a primitive is when the entire range of values for the primitive is not valid for an argument. Enumerations are a good example of such an encapsulation, as they are really just a subset of integers mapped to meanings. As an aside, Java allows the addition of methods and thereby a kind of polymorphism to enums making them superior to C# enums.

Strings are always objects. This has different memory implications for managed vs. unmanaged languages. Type safety for argument passing and range of values both apply here as well, though obviously not mathematical operations. A good example is file system paths. Several characters are not allowed in file system paths, and the object itself should enforce such restrictions.

Rule 8: Use first class collections

Rule 8 should be more than an exercise. A mutable collection should almost never be passed between classes without being wrapped. Any operations performed on the collection should be extracted into methods and added to the wrapped collection. Encapsulating collections adds minimal coding and memory overhead to most languages, and greatly increases readability and encapsulation.

Applying the idea of calisthenics to programming is an interesting idea. As with physical calisthenics, object calisthenics are helpful to the entire spectrum of developers.

ReSharper Next

2009-06-04T08:33:00.001-07:00

ReSharper (or IntelliJ) is an invaluable tool when programming in C# (Java). Proficiency in ReSharper increases productivity by allowing rote refactorings to be performed with a couple of keystrokes. For languages such as C# or Java where ceremony is high, such tools greatly lower the amount of work required to keep code maintainable. I am very happy with ReSharper but I can't help but want more. I think this is a sign of a very good product. It does what it does very well, and therefore I trust Jetbrains to be able to add more features with the same elegance.

Coverage

I recently got an email from the developers at NCover telling me a new version is ready (a complete rewrite no less). Unfortunately, NCover has yet again missed out on the killer feature for coverage: IDE integration. My favorite version of NCover was the open source version that I could run via TestDriven.Net from the IDE. While bitching about having to pay for a version of NCover I find inferior to the free version, a coworker brought up an even better point. Not only do I want IDE integration, I want it to color my source code in Visual Studio. I want it to slap me in the face with the fact that I have written a line of code that is uncovered. Here is a mocked up example:

Given how well ReSharper has done with IDE integration (including unit testing) it would be nice if they would include integrated coverage in the next version as well.

Metrics

Everytime I check in code to the continuous integration server a set of metrics (Cyclomatic Complexity, Lack of Cohesion of Methods, Depth of Inheritance, Coverage, and Duplication) are run. These are arranged into a convenient chart available via a web interface. However, I have to actually navigate to the web interface to see these reports, which makes them easy to ignore. What would be better is if the feedback loop were shorter and I knew while I was working on the code that my Lack of Cohesion of Methods was too high indicating a potential violation of the Single Responsibility Principle. As with coverage, the results of these metrics being right in front of me while I am coding would have a positive impact on the maintainability of the code.

Advanced Live Templates: Test Data Builder

I recently discovered the pattern of Test Data Builders to help in object construction for testing. It is particularly useful if the object has a lot of constructor arguments, many of which do not change between tests. Test Data Builders require a decent amount of typing and most of it is rote and could be automated. My first thought was to write a custom ReSharper live template. Unfortunately live templates are not powerful enough to do this. One could write a class to code generate it, but it doesn't quite have the same feel as a 'CTRL-T-D-B'. I want to be able to write C# code in live templates, or at the very least provide a built-in template for Test Data Builders.

ReSharper is an excellent tool. I could not imagine coding in C# (or Java with IntelliJ) without it. There is little improvement left for the current feature set. Integrated code metrics and full C# support in live templates would be very beneficial features for the next iteration of ReSharper.

Agile True Believers

2009-03-28T11:21:00.001-07:00

I recently attended SpeakerConf, a conference with the intention of only inviting speakers of software related conferences with the idea of "capturing the essence of those productive conference dinners, without having to deal with the rest of the conference" (I'm paraphrasing). I have never spoken, nor attended, a technical conference, but I do work with one of the organizers Jay Fields, and he was gracious enough to invite me without having ever spoken at a conference. I learned a good amount at SpeakerConf, and more importantly met many people I otherwise would not. I even had the honor of bouncing my ideas on inheritance off of Michael Feathers.

At SpeakerConf a speaker explained a term called FDD or Feedback Driven Development, which reminded me of a previous post I wrote about searching for more accurate terms for how I develop. I started to write that post with a different goal in mind than when I started it. The original intent of the post was to criticize how I found people using Agile as a catch-all term to describe any development related practice they favored. Because Agile is such a nebulous term in this context, I questioned the use of a single term to describe a set of nuanced practices. By the end however, I just came up with a different term, feedback, and in retrospect I simply replaced one term with another, contradicting the intent of the post. I have been collecting my thoughts about this topic for a while and finally have my thoughts together enough to correct my stance.

While I adhere to many of the development practices espoused by this group of well known Agilists, I disagree with many of them on the Agile movement itself. Mass movements come with zealots, and in my opinion zealots are an easy way to turn off the developers that I want to work with. Many of them felt zealots were a good way to browbeat the stubborn developers into conversion to Agile development practices. When it comes to zealotry in the Agile community, Uncle Bob makes this distinction on his blog:

"There is a difference between a zealot, and a religious fanatic. A religious fanatic cannot envision themselves to be wrong. We in the agile community may indeed be zealots, but we know we can be wrong."

Um, really? There are no "religious fanatics" (a poor choice of words) in the Agile community? Agreed there are rational people in the Agile community, but to say that all of them argue rationally (we know we can be wrong) instead of emotionally is not true. Herein lies the danger behind using a mass movement. Mass movements are often propagated by not just zealots, but what Eric Hoffer calls True Believers. Below is my definition which I derived from his book.

True Believer - n. a person who joins a mass movement due to lack of confidence in one's abilities and intellect. By adhering to a simplified and known set of dogma the person becomes a member of a larger entity, providing social camaraderie, a sense of importance, and most importantly lack of personal accountability for failure. The zeal of the True Believer is proportional to the irrationality of the doctrine.

Does the existence of True Believers discredit mass movements in general? Of course not. But it does alienate those of us who are turned off by irrational argumentation that is founded on authority, emotion, or other logical fallacies instead of valid evidence. In fact, many movements are viciously and exclusively defended by fanatics because their doctrine doesn't hold water. Ideally critical thinkers would investigate deeper, but between skepticism being an essential component of scientific thought, and the sheer amount of utter bullshit espoused by our fellow man, it is understandable that some critical thinkers write off some mass movements without due diligence.

Agile is not the only victim of True Believers. GTD (note David Allen is an ordained minister of a new age religion) suffers from the very same problem. Richard Dawkins, in The God Delusion, purports that churches and religious summer camps should be replaced with organized atheist-friendly political groups and "skeptics camps" for children. Is there really a difference? Did Dawkins miss a main motivator behind not having a religion? The New York Times recently had an op-ed on how the scientific community should stop using the term Darwinism when referring to biological evolution because it associates an otherwise strong scientific theory with an -ism and an idol.

What wouldn't spurn away those who think critically? It is here I would like to return to the initial intent of my original post. Perhaps marking a way of developing software with a single term like Agile or Feedback is a gross oversimplification. Instead of mass movements in the software community one could be more granular and debate about the merits of individual practices instead of arbitrary groups of them. The problem is inherent in coming up with a comprehensive yet numerable set of techniques or mantra for any sufficiently complex intellectual endeavor. The list will become either overly proscriptive and mechanical with no room for interpretation or will be small and nebulous to the point of being meaningless. I vote to let each practice stand on its own, without a flag or army, and especially without the pitchforks of the True Believers.

Application.Run and Controller-View Separation

2009-02-21T06:22:00.001-08:00

When starting a new WinForms application, Visual Studio provides you with some defaults to help you get you started. Two of these are the Program.cs file and a child of the Form class as the project's first window. In the Main method of Program.cs, you will see the following code:

Application.Run(new Form1());

It is here that Microsoft starts you off on the wrong foot. The natural place to start, given this setup, is to add logic to the Form (which is the View). By making the View the starting point of the application, proper architecture is discouraged from the beginning.

All is not lost however. There is an overload for Application.Run that takes an ApplicationContext. Simply create a Controller, that extends ApplicationContext, and have it take a reference to the View. This design encourages logic to be placed in the Controller which can then drive the View. Easier unit testing is one of many reasons such a design is superior.

// Controller.cs
public class Controller : ApplicationContext
{
    private readonly Form1 _view;

    public Controller(Form1 view)
    {
        _view = view;
    }   
}

// Program.cs

// in Main
Application.Run(new Controller(new Form1());

This simple change in default setup has a dramatic architectural effect on the application going forward. While many experienced developers will invert the default relationship, as shown in the snippet above, it leads novice .NET developers down the wrong path unnecessarily. Why make a superior architecture available but not make it the default?

The Costs of Emotional Immaturity

2009-02-15T10:21:00.001-08:00

I was following the XHEO controversy recently on Ayende's blog when I happened upon a comment instructing the XHEO executive to learn something about customer service from Joel Spolsky. I find Joel to be hit and miss, especially when it comes to technical content and workspace layout. However, his entry on customer service is spot on. In particular I find the following passage very important:

“Ah. It’s my fault,” he said.
And suddenly, I wasn’t mad at all.
Mysteriously, the words “it’s my fault” completely defused me. That was all it took.

For some reason, some adults have a very difficult time admitting they are wrong, especially in front of a group. This is perplexing, all people make mistakes, to not admit when one is wrong is delusional, and therefore I would argue immature and unprofessional. Stubbornly holding a position against a group of adults who know that the offender is attempting to save face instead of arrive at the best solution is not a good way to earn the respect of those in the room, regardless of age, tenure, or rank.

Not only is it a sure way to lose respect, it is a timesink. When such an incident happens in a meeting, the offender generally must be pulled aside individually so their ego can be placated in a one-on-one environment. Then a third meeting must be called with the original group where the original solution is covered again, except the offender does not raise any objections this time, thereby demonstrating consensus. This is a waste of two entire meetings and everyone's time who has to attend. All because one person isn't comfortable publicly admitting they are human.

These timesinks add up over time. My current group doesn't experience this problem at all, and our velocity is noticeably higher than groups that refuse to mitigate the problem. The culture is to admit when one is wrong, publically if that happens to be the context at the time, as soon as one realizes it. We work on very difficult problems, leading people to be wrong frequently. This is expected, and because everyone in the group is emotionally mature enough to admit it immediately, we discard subpar solutions as quickly as possible. If anything, we all respect each other more because we readily admit when we are wrong.

Not every discipline has this luxury. Software development is a fairly objective area of study. CPU and memory footprint, testability, and maintainability are all measurable, so long as the group agrees on the definitions. Leverage the relative objectivity of software development to your advantage. Encourage a scientific culture, one where skepticism, dissent, falsifiability and correctness are emphasized above shielding everyone's inner child. Next time someone proves your stance incorrect logically or empirically, think twice before arguing from emotion. It might save the group time and increase your respectability amongst coworkers.

Unit Testing GUIs

2009-01-31T14:57:00.001-08:00

A couple of years ago I was doing mostly GUI programming. At around the same time I starting getting into unit testing. Unit testing soon became an invaluable way to validate my implementation and drive my design, but I found it difficult to apply to GUIs. I scoured Amazon and Google trying to find anything that would help me out. Most of what I found was along the lines of "GUI testing is a hard problem that is largely unsolved." Very discouraging results.

After a while, I did find one pattern that looked like it had potential: Passive View. The original web page is a little rough and difficult to read, but after going through it a couple of times it clicked (later I found an easier to read version of same pattern which the author calls Presenter First).

Passive View is a version of Model-View-Presenter. The distinction between Model-View-Presenter and Model-View-Controller depends on whom you ask; you can follow the Passive View link if you are curious. Passive View is different from other implementations of MVP in that there is no coupling between Model and View. The goal of Passive View is testability. If you are unsure if you are properly following Passive View, ask yourself "Is what I am about to code difficult to test?" If the answer is yes, then you are not following Passive View. Below are the guidelines to follow for Passive View:

Keep the View as thin and logic free as possible.
Put as much of the presentation logic as possible in the Presenter instead of the View.
Only the Presenter should reference the View.
The View should not reference any other objects, including the Model.

The View should implement an interface on which the Presenter should depend. This allows for substituting the real View with a test double view while testing the Presenter, via Constructor Injection. Because the vast majority of the implementation is contained in the Presenter, the GUI code is now easy to test. Below is an example using C# 3.5 and Rhino Mocks 3.5.

public interface IView
{
    FlexGrid Grid { get; }
    event RowColEventHandler CellChanged;

    string SaveButtonText { get; set; }
    bool SaveButtonEnabled { get; set; }
    event EventHandler SaveButtonClicked;
}

public partial class MyView : Form, IView
{
    private Button _saveButton;
    private FlexGrid _grid;
    
    public FlexGrid Grid
    {
        get { return _grid; }
    }

    public Color SaveButtonColor
    {
        get { return _saveButton.BackgroundColor; }
        set { _saveButton.BackgroundColor = value; }
    }

    public bool SaveButtonEnabled
    {
        get { return _saveButton.Enabled; }
        set { _saveButton.Enabled = value; }
    }

    public event EventHandler SaveButtonClicked
    {
        add { _saveButton.Click += value; }
        remove { _saveButton.Click -= value; }
    }
}

public class MyPresenter
{
    private readonly IView _view;
    private readonly IModel _model;

    public MyPresenter(IView view, IModel model)
    {
        _view = view;
        _model = model;
        _view.SaveButtonClicked += SendChangesToModel;
        _view.CellChanged += EnableSaveButton;
    }

    private void SendChangesToModel(object sender, RowColEventArgs e)
    {
        _model.ChangeInfo(_view.Grid[e.Row, "Author"], _view.Grid[e.Row, "Book"]);
        _view.SaveButtonEnabled = false;
        _view.SaveButtonColor = Colors.Gray;
    }

    private void EnableSaveButton(object sender, EventArgs e)
    {
        _view.SaveButtonEnabled = true;
        _view.SaveButtonColor = Colors.Firebrick;
    }
}

Notice button properties are delegated-to in the View, but with the FlexGrid only events are redirected. This is because if the button were exposed directly or through an adapter with an interface so it could be mocked, one would have to create two mocks and have one return the other. It is easier to "denormalize" as many of the properties as possible to one interface. The exception to this technique is with a heavily normalized and complex class like FlexGrid. It would be very cumbersome to delegate all of the methods to an adapter class or on the Form. Instead a real FlexGrid is returned, generally one generated by the real Form so that it is properly configured (columns are already set up). The same could be done with the Button, but not being able to use the events on the button itself is confusing, so the previous technique should be used only if necessary. Neither event is subscribed to directly because it is required to be able to fire these while testing. Putting the events on interfaces allows them to be fired with Rhino Mocks like so:

[TestFixture]
public class MyPresenterTests
{
    [Test]
    public void ButtonClickShouldSendChangesToModel()
    {
        var view = MockRepository.GenerateMock<IView>();
        var model = MockRepository.GenerateMock<IModel>();
        var grid = new MyView().Grid;
        view.Stub(v => v.Grid).Return(grid);
        var underTest = new MyPresenter(view, model);
        grid[1, "Author"] = "Carl Sagan";
        grid[1, "Book"] = "A Demon Haunted World";
        view.Raise(x => x.SaveClicked += null, view, new RowColEventArgs(1, 0));

        model.AssertWasCalled(m => m.ChangeInfo("Carl Sagan", "A Demon Haunted World");
        view.AssertWasCalled(v => v.SaveButtonEnabled = false);
        view.AssertWasCalled(v => v.SaveButtonColor = Colors.Gray);
    }
}

Lastly if a view or presenter gets too large but you want to keep the window together, simply break up the interface into pieces and have the same Form implement all of them. Lastly create smaller presenters that have a common presenter connecting between them.

public interface IGridView
{
    FlexGrid Grid { get; }
    event RowColEventHandler CellChanged;
}

public interface IControlPanelView
{
    string SaveButtonText { get; set; }
    bool SaveButtonEnabled { get; set; }
    event EventHandler SaveButtonClicked;
}

public class MyView : Form, IGridView, IControlPanelView
{
    // implementation left out for brevity
}

public class GridPresenter : IGridPresenter 
{
    private IGridView _view;

    public GridPresenter(IGridView view)
    {
        _view = view;
    }

    // implementation left out for brevity
}

public class ControlPanelPresenter : IControlPanelPresenter
{
    private IControlPanelView _view;

    public ControlPanelPresenter(IControlPanelView view)
    {
        _view = view;
    }

    // implementation left out for brevity
}

public class CommonPresenter
{
    private IGridPresenter _grid;
    private IControlPanelPresenter _controlPanel;

    public CommonPresenter(IGridPresenter grid, IControlPanelPresenter controlPanel)
    {
        _grid = grid;
        _controlPanel = controlPanel;
    }

    // implementation left out for brevity
}

Why is CreateHandle protected?

2008-12-20T09:00:00.000-08:00

In WinForms, there is a method on Form called CreateHandle that is protected. The method simply forces the Form to create a handle. An exception is thrown if you attempt to modify the form before a handle is created.

In most cases calling CreateHandle is not necessary. When you open the Form, a handle is created automatically. However there are times when I want to modify the Form before the user opens it. To do this I have a couple of choices. I can inherit from Form and expose CreateHandle in a new base class. I can individually expose CreateHandle in every Form for which I want to have this ability. My last choice is to use reflection to gain access to the protected method. Some people will mention that you can simply query the Handle property and that will create a handle. This is not quite true, when using this technique a handle is not created for children of the parent Form:

http://msdn.microsoft.com/en-us/library/system.windows.forms.control.createcontrol.aspx

I do not like any of these options. In fact this is an example of why I do not think protected is a valid level of encapsulation at all. If the functionality is useful, make it public. If it breaks encapsulation, then make it private. Protected is an annoyance that sort of sits between encapsulated and not. Why favor inheritance, a technique that is usually inappropriate and brittle, when composition would be a better choice. The use of the protected keyword pushes me towards inheritance as a design choice even though it is inferior.

If I were to design an object-oriented language, I wouldn't include the protected level of encapsulation.

Rhino Mocks 3.5

2008-10-22T19:40:00.000-07:00

I had a chance to upgrade to Rhino Mocks 3.5 recently. While I won't go over all of the new features I will go over the ones I have found most useful.

Static creation methods

CreateMock is deprecated. Use StrictMock instead. StrictMocks create brittle tests and therefore usage is discouraged. A better route is to forgo using an instance of a MockRepository altogether. Instead two static creation methods are available:

var mock = MockRepository.GenerateMock<IMock>(); // same as DynamicMock
var stub = MockRepository.GenerateStub<IStub>()

Compare with:

MockRepository mocks = new MockRepository();
IMock mock = mocks.DynamicMock<IMock>();
IStub stub = mocks.Stub<IStub>();

Note the terms "Mock" and "Stub" instead of DynamicMock and Stub. This is to be more in line with popular test double terminology which carries the benefit of being more intention revealing. Mocks should use Expect with constraints (using Arg<T>) before exercise of the unit-under-test (and VerifyAllExpectations after) or AssertWasCalled and AssertWasNotCalled after the exercise. Stubs should use the Stub method to return canned data which should be called before exercising the unit-under-test. Both Stub and Expect can be used on mocks, if you want to mix and match, but it makes the code less clear. Only Stub can be used on stubs. All properties on Stubs are defaulted to UsePropertyBehavior as with the old API. Note that Stubs cannot make the test fail.

Goodbye record-playback

The explicit phases of record and playback (and the using scopes along with them) is no longer necessary. The idea here is to simplify mocking. The new technique is called Arrange, Act, Assert or AAA for short.

Arrange - set up the unit under test
Act - exercise the unit under test, capturing any resulting state
Assert - verify the behavior through assertions

This is a long-winded way of saying exercise the unit under test and then verify the results, effectively making unit tests that use test doubles as succinct as those that don't.

Inline constraints

var view = MockRepository.GenerateMock<IPassiveView>();
// exercise unit-under-test
view.AssertWasCalled(x => x.SetSaveButtonState(
   Arg<string>.Matches(t => t.StartsWith("Saved")),
   Arg<bool>.Is.Equal(true),
   Arg<Color>.Is.Anything));

var view = MockRepository.GenerateMock<IPassiveView>();
view.Expect(x => x.SetSaveButtonState(
   Arg<string>.Matches(t => t.StartsWith("Saved")),
   Arg<bool>.Is.Equal(true),
   Arg<Color>.Is.Anything)).Return(true);
// exercise unit-under-test
view.VerifyAllExpectations();

This used to require creating a separate custom delegate for each different signature and then an assertion for each argument within the callback. The new syntax saves a lot of keystrokes. Note that with assertions no explicit verification is necessary. If there are expectations, they are to be verified on a per mock basis. This encourages using few mocks, however if multiple are used, the granularity of verification is smaller and cleaner. This was available in the previous API but had an uglier syntax before extension methods.

New event raising syntax

I find this new API call particularly useful with the PassiveView pattern to simulate the firing of UI events.

view.Raise(x => x.SaveClicked += null, view, EventArgs.Empty);

Compare with:

view.SaveClicked += null;
IEventRaiser buttonClick = LastCall.IgnoreArguments().GetEventRaiser();
buttonClick.Raise(view, EventArgs.Empty);

WhenCalled instead of CallBack

WhenCalled is a cleaner replacement for CallBack. It allows access to all of the arguments and the return value through the invocation object. It should not be used for validating arguments, use the Arg constraint instead. It is useful for keeping what is passed in or setting certain state when a particular method is called. For example:

var wasFired = false;
var buttonText = string.Empty;

view.Stub(x => x.SetSaveButtonText(Arg<string>.Is.Anything))
   .WhenCalled(invocation =>
      {
         buttonText = (string)invocation.Arguments[0];
         wasFired = true;
      });

The new 3.5 syntax is vastly superior to an already useful API. It is more succinct and expressive, lowering the barrier for entry and making code much more readable for the maintainer of the tests.

SVN Renames

2008-09-27T07:38:00.000-07:00

I am a huge fan of Subversion. Maybe it is because the first SCM I used professionally was StarTeam, for which gzip and email provides a better solution, but that is probably a different discussion.

To be honest it is probably because svn provides atomic commits, cheap copies, a good CLI, has a single responsibility, optimistic locking, has tools that provide good IDE and file system integration, is free, and my favorite feature svn:externals which saves immense amount of time when used for binary dependencies.

Every piece of software has its warts, and svn is no different. While good at branching, historically svn has lacked an easy way to merge. Version 1.5.0 introduced merge tracking to allow for easier merging.

I was really looking forward to this feature, not because I branch a lot, but when I do, creating pre and post tags is tedious. I usually take care of this problem with a Ruby script, but a solution that works well with IDE and file system integration would be nice.

As noted by the svn developers themselves, merge tracking is not complete as of 1.5.2, however one bug in particular is a deal breaker for me. Merge tracking does not play well with renaming files.

The problem with renames in svn annoys me the most. In svn renames are not atomic. Instead they are composed of a copy and delete. It is unfortunate that the request for fixing this problem early was dismissed.

At this point, most developers are probably saying, svn is open source, go check it out and fix it yourself. Well I did check out the svn code and I looked in to fixing it. But between the complexity of the change (it requires a change in the schema) and my rusty C skills, I don't think I have much of a shot of pulling it off.

So this is a call to the svn developers out there. Please fix the rename problem and the possibly related merge tracking problem soon. I rename classes and subsequently files frequently and while the lack of atomicity normally has not gotten in the way, it is a roadblock to adopting the 1.5.x line and subsequently taking advantage of merge tracking.

Agile -> Lean -> Feedback

2008-09-20T08:22:00.001-07:00

A colleague of mine recently lampooned the Agile (proper) movement. The explanation behind his iconoclasm was eye-opening. In his presentation he listed a set of practices that our group considered Agile such as continuous integration, unit testing, acceptance testing, separation of concerns, discouraging singletons, sprints, stories, etc. Right next to this list he displayed the Agile Manifesto. There was a serious disconnect. Not so much that the lists conflicted, as much as the Agile Manifesto is wildly open to interpretation.

While certainly a step forward when it originated, Agile has become a brand name in the development community, and after the aforementioned presentation, I am not sure if it is meaningful in and of itself. Agile has become the software equivalent of the Interstate Commerce Clause. Unfortunately due to its popularity and nebulous definition, Agile is sometimes used to label any practice that an advocate finds desirable. Opponents are then often written off for attacking an Agile practice, instead of on the merits of the actual criticism.

In an effort to move back to analytical review, instead of emotional attacks by True Believers, I looked for a new label to use in place of Agile that is more descriptive of my favorite practices. Lean is a newer label adopted from Toyota manufacturing that comes closer to getting the job done. Waste is the ultimate evil and continuous monitoring and elimination of waste is paramount to Lean development.

Like most attempts to make parallels between traditional engineering disciplines and software development, I think Lean also comes up short. This is due to the significantly higher level of complexity of software and the subsequent organic/iterative process that is required to develop it.

After examination of the practices listed above that I find most useful, I think "shortening the feedback loop" is a better mantra. Pair programming introduces code inspection as the code is written. Unit tests give the developer design and correctness feedback as the tests are written, and maintainers feedback when they break current functionality. Continuous integration lets the developer know as soon as possible that the build does not compile, the tests do not pass, or a metric has breached a threshold. Stand-ups and retrospectives find impediments early and provide a means to fix them. Sprints and stories give the stakeholders feedback on what they thought they asked for, and how long it will take to finish.

"Shortening the feedback loop" is a much more intention revealing goal than "Agile". It disarms some of the religious fervor of modern software practices, clarifies what is useful, and gives helpful practices a more analytical base on which to stand.

History Is Written By The Victors

2008-08-24T10:09:00.000-07:00

Every once in a while, the software field makes a wrong turn. One of these decisions was mutual exclusion over message passing for concurrency. I contend another is C++ over Obj-C. C++ is taught in universities and is arguably the de facto general programming language in the field. Obj-C is used by Apple. I always found it strange that Apple, during its scariest period, decided to go with the unpopular Obj-C over the very popular C++ as the language of choice on its brand new platform. It turns out this decision was made at NeXT, the company who wrote the predecessor to OS X. But why did NexT choose Obj-C over C++? Maybe they made the right decision...

What are the benefits of Obj-C over C++? The primary reason is that Obj-C is a simple extension to C, a language that is also simple. C++ is however far from simple. C can be picked up in a weekend. Mastery of the arcane C++ requires no less than all of these.

The designers of Obj-C also made some design decisions that are superior to those of C++. Obj-C does not allow multiple inheritance or operator overloading and has dynamic typing. It also has categories (like extension methods in C#), protocols (like interfaces in Java/C#), and reflection. Reflection is almost indispensible for dependency injection, test discovery, and UI development.

To boot, Obj-C 2.0 allows for optional garbage collection, although this is on the C++ road map as well.

However, as usual, Obj-C is not perfect. Obj-C lacks namespaces entirely. In fact most classes start with NS, such as NSObject, which stands for NexTStep to obviate the need for namespaces. Another oddity is that there is no private modifier, only public and protected, making the Open-Closed Principle difficult.

Perhaps a more modern question than Obj-C vs. C++ is Obj-C vs. Java. Java is a completely managed language, eliminating the common security issue of buffer overflows. Also, Java is JITed via a virtual machine, allowing for both cross platform execution and run-time optimization. Obj-C being an extension of C provides neither of these benefits. However what Obj-C does provide is the potential speed of C (when C is actually faster), but with abstraction available when desired. Garbage collection is optional, dynamic typing is available, but at the same time the range checking of managed languages is not mandatory, and one has access to libraries and hardware interfaces (like OpenCL) written in C without having to pay the cost of language interop.

If you write Obj-C applications for the Mac, you also gain all of the benefits of the Cocoa API. This gives you access to Core Animation, Core Image, Core Video and a framework that all but mandates model-view separation unlike some others. With Cocoa, cutting edge UI development is possible with Obj-C.

Should you consider Obj-C over C++? The gcc compiler supports Obj-C, so you aren't restricted to the Mac platform, though you won't have access to Cocoa (though GNUStep is available). Also many of my favorite tools, such as an xUnit framework, are not available in Obj-C. The switch is certainly worth considering though, and with the Mac platform doing so well I anticipate the Obj-C community growing.

The B Team

2008-08-07T11:31:00.001-07:00

Jay Fields, in a recent post titled Elephant in the Server Room, stated that 50% of software developers should find a different line of work. While this may sound elitist, as some have pointed out, I think Jay's post has merit. The reason is that the spectrum of talent in software development is very wide. Robert Glass even attempted to quantify it:

"The best programmers are up to 28 times better than the worst programmers, according to 'individual differences' research. Given that their pay is never commensurate, they are the biggest bargains in the software field."

- Robert Glass, Facts and Fallacies of Software Engineering

28 times. That is over an order of magnitude. Professionally, I have only worked as a software developer, but I find it hard to believe that such a difference exists in other fields. For instance, how many other professions have a website dedicated to pointing out just how bad some coworkers are?

For added effect, here is an anecdote I experienced myself with a former coworker:

I worked with a developer named John (name changed to protect the guilty). John had a favorite interview question he would ask candidates:

"How do you reduce the number of buckets in a hash table?"

Now most of us that passed our sophomore year in college know that this is a silly question. However John insisted that reducing the number of buckets in a hash table "made it faster." It did. Let me explain how. John was trying to hash up to 10,000 objects each with a unique number from 1 - 10,000. Most of us would stop right here and just use an array in order to gain constant access. John however decided he needed a hash table. He noticed hashing all of these objects was very expensive. Instead of realizing he was breaking the load factor and initial capacity and adjusting these accordingly in the constructor, he decided to override GetHashCode() as follows:

public override int GetHashCode()
{
   return base.GetHashCode()%256;
}

So basically he got 10,000 entries to collide over and over again on the first 256 buckets, thus providing O(n/256) or O(n). Not quite the O(1) hash tables usually provide. Notice he even used a non-prime number, the worst numerical characteristic for the number of buckets in a hash table.

Upon pressing John about why his "optimization" "worked" he replied "I do not know Microsoft's implementation." He was Programming by Accident and didn't even know how a hash table worked. We almost lost a good candidate due to this bogus question. Additionally, even after the concept of hash tables was explained to John, his "optimization" was still found in code a year later.

Unfortunately, as the DailyWTF shows, this is a common occurrence. More importantly this lowers the morale of the developers you actually want to keep around. Bruce Webster talks about it in The Dead Sea Effect.

The lowering of morale is not limited to software. Jim Collins attributes the loss of morale to an increase in bureaucracy that is necessary to manage the less competent.

"Most companies build their bureaucratic rules to manage the small percentage of wrong people on the bus, which in turn drives away the right people on the bus, which then increases the percentage of wrong people on the bus, which increases the need for more bureaucracy to compensate for the incompetence and lack of discipline, which then further drives the right people away."

- Jim Collins, author of Good to Great

The gap in talent in software is vast and Jay is not the first to point this out. Helping some of the lower tiers find new employment might help the industry advance quicker and make the lives of the talented less stressful.

Avoid Singletons

2008-08-05T19:14:00.001-07:00

In part II of the "avoid" series, I am going to discuss the Singleton. Singleton is not a pattern, it is an anti-pattern. Singleton is the modern equivalent of global methods/variables in C/C++. It is diametrically opposed to maintainability because it introduces a strong coupling between classes, the very same problem that IoC does an excellent job of solving. For all the good the Gang of Four did with their canonical patterns book, they really shat the bed with the Singleton pattern.

By Singleton I mean the kind with a static GetInstance() method. My favorite part of interviews is when the candidate thinks they are being impressive by letting me know the double locking gotcha involved with the GetInstance() method. In reality they have simply notified me that they write code that I would not want to maintain. An acceptable way to use Singletons is to remove the static GetInstance() method, and instead inject the class with an IoC container. By using an IoC container, one can still retain singleness while eliminating all the problems with the classic implementation of the Singleton.

Please if you write code, stop using this "pattern". Tell your coworkers the same.

Avoid Inheritance

2008-07-24T12:04:00.000-07:00

There are three classic properties of object orientation. Encapsulation, inheritance, and polymorphism. At least this is what was drilled in to my head in school. Over the last several years, I have strived to use all three to make my code more object-oriented, however inheritance is one that I found was being overused, both by myself and others. The following passage from Practices of an Agile Developer really brought to light the problem:

"When using inheritance, ask yourself whether your derived class is substitutable in place of the base class. If the answer is no, then ask yourself why you are using inheritance. If the answer is to reuse code in the base class when developing your new class, then you should probably use composition instead."

The problem was that I was using inheritance mainly for reuse. The code I wrote usually passed the 'is-a' test, but rarely would I use the child class via a reference of the parent class's type, known as polymorphic substitutability.

I have since avoided inheritance unless polymorphic substitution is desired and found that I now almost never use inheritance. Composition, or delegation, usually provides a much better alternative, although in languages without mixins (like Ruby), it requires more code. Note that mixins in Ruby (or Extension Methods in C#) do not completely solve the problem as they cannot contain state themselves. I prefer maintainability above code brevity, so for me the choice is easy. If you use a tool like ReSharper (which includes a 'delegate-to' shortcut) the typing overhead is negligible.

To be clear inheritance does not include interface implementation. Java separates these two concepts by using two separate keywords: 'extends' for inheriting from a class, and 'implements' for implementing an interface. In C# both words are replaced by the less expressive ':' but interfaces by convention start with an 'I'. Both Java and C# use separate 'interface' and 'class' keywords and allow different behavior for each (multiple vs. single).

C++ however is the worst offender. It lacks a convention for interfaces, it lacks separate syntax for implementing vs. extending, and most importantly it lacks different keywords for interfaces and classes. I conjecture that this has lead some C++ developers, myself formerly included, to not realize that extending a class and implementing an interface have vastly different maintenance implications. By not treating interfaces and classes separately, it also required the inclusion of multiple inheritance and other warts such as virtual inheritance that would have been otherwise unnecessary. C++ does allow for private inheritance, which provides for reuse without being an 'is-a' of the parent, but I fail to see how this is superior to delegation. Delegation at least provides some explicit scoping.

So why does improper use of inheritance lead to less maintainable code? The reason is that inheritance works against encapsulation. Even if the Open-Closed Principle is closely followed the child class is strongly coupled to the parent class. This makes testing the child class in isolation from the parent class impossible in most languages. If the maintainer of the child code is not working closely with the maintainer of the parent code, the child code is now strongly coupled to the whims of the parent implementation. None of this is necessary with delegation (if the delegated-to-class is injected). Protected methods make this worse. These are special extension points that are not exposed to the outside world, only to child classes. Often times the Template Method pattern is used here where a parent method calls abstract child methods to complete the behavior of the parent class. This pattern looks very slick as it is quite DRY, but it leads to improper intimacy of objects creating problems when either class is changed or needs testing. A better solution to Template Method is to inject the former child class in to the former parent class. The former parent class can then call the former child class when it needs a specialized implementation, but is only bound to the former child's interface.

Polymorphism, through interface implementation, is very powerful. It implies that the class provides a "service". Having a class reference another class through an interface decouples them, allowing the referencing class to know very little about the referenced class. In fact the referenced class may be entirely replaced without the referencing class being changed at all. Depending on interfaces instead of classes allows for decoupled, maintainable code.

So why was such an easily misused and rarely helpful feature considered one of the pillars of object oriented programming? I would have preferred mixins or dynamic typing over inheritance. The less popular Obj-C has inheritance, but only single, and the language includes dynamic typing. In fact in dynamically typed languages, where polymorphic substitutability is free, inheritance is almost completely inappropriate. However you will still find it being used by respectable developers.

UI Dos and Don'ts

2008-07-06T11:38:00.000-07:00

For some reason developers think it is difficult to create decent User Interfaces. Their difficulty is evident in the high percentage of UIs found in the wild that are difficult to use. It is thought that a level of artistry and aesthetic sense is required. It is true that this helps, but it is not required. In fact there are a few simple rules I have gleaned from my favorite two UI books: Humane Interfaces by Jef Raskin and Designing Interfaces by Jennifer Tidwell that will get a UI most of the way there.

Avoid modality

This is the classic message box. The problem with message boxes is that they are informational only. There is no decision to make. All the user can do is read the message and dismiss the box. Modal dialog boxes interrupt the user work flow, and cause a user "context switch". Instead put the message on the status bar, or in the above case, it would have been better to outline the recipient textbox in red with the error message underneath.

A similar case is the "Are you sure?" "Are you really sure?" "Absolutely?" modal window. Yes I am fucking sure. That is why I clicked the original action in the first place. Ok so maybe I didn't. Maybe I accidently used a short cut key to close the program and I haven't saved a document yet. If the action is undoable, there is a solution. Allow undo. Yes it is more work, but it makes a better UI.

Here is an example of two UIs used to find text. The top one pops up a new window over the text the user is currently reading again interrupting their train of thought. The bottom instead creates a narrow area that lines up along the bottom of the text, probably away from where the user is reading. It then highlights the text where the user was concentrating. The bottom example is superior because it doesn't break up the user work flow by avoiding modality.

User data is sacred

Often times error correction is a necessary part of the user interface. However, even if the user inputs invalid data, one should not throw it out like the example above. The user may have worked hard for that data, or forgot what they typed to upset the software. Instead leave the data they typed in and mark it as incorrect. Then as they correct it themselves, unmark the input. See below.

Habituation

Above are two infamous user interface patterns from MS Office. The first is what was known as a "personalized menu." These break what is known as habituation or the formation of habits. This is because the menus change on the user as you use them which lengthens the time before the location of the menu becomes a habit, thus slowing down the user and requiring more thought.

The second example is the improved UI from a habituation standpoint. The initial reaction by many users to this new UI was poor. It is because it is radically different from the previous UI and thus broke habituation. However, after becoming acquainted with the new interface it is vastly superior as every item is entirely stationary. There are no more menus and the items never appear over the document. Also, they are arranged from left to right and top to bottom (the way English is read) in order of prevalence of usage during user testing. Size of the targets is also an indicator of prevalence of usage during user testing. After getting used to this new UI, users are significantly more productive.

Another common way of breaking habituation is overriding common shortcut chords. For example do not override ctrl-x, ctrl-c, or ctrl-v.

Hues and tints

The example on the left is slightly contrived, but I have seen examples of such software, I just didn't feeling like taking a screen shot at the time. The idea here is to pick a color or two (a hue) and then vary other UI elements by various tints on those few hues. The left window picks too many contrasting hues and no tints which causes cognitive dissonance. Jennifer Tidwell calls this "fruit salad". It reminds me of early user pages on My Space.

This may seem to contrast with the suggestion above regarding coloring incorrect fields, but it does not. High contrast is useful to get the users attention. But if there are multiple high contrast items, then the value of contrast is lost and all items become distracting and painful to view.

Edges and corners

This may seem like a given, but I see many "non-UI" developers forget to properly line up and anchor elements. Make sure to take the time to line all of the widgets up, and create areas of interest that resemble symmetric, simple shapes (such as rectangles). Most UI builders help the designer with this task. Also make sure to eliminate excessive blank space. Make sure the widgets span almost the entire control, with small equal margins on all sides.

Maintain user focus

Above there are two windows that are nearly the same. They differ in how the buttons are labeled. The first has the usual OK and Cancel, the latter actually describes what the buttons do. The latter is superior because the user is performing the following steps: (1) reading the message to figure out why the software needs the user to make a decision, (2) moves attention to the buttons to make the decision. By putting descriptive labels on the buttons, you allow the user to remain focused where they are clicking, instead of rereading the message to figure out what they are OKing or Canceling.

Another example is the deeply nested windows required to configure wireless networking in Windows XP. It is possible to get 5 to 7 windows deep and at times making changes at all of these levels. This requires the user to jump between areas of focus to get a full glimpse of the configuration. Not a good idea.

This list is by no means exhaustive, but touches on the majority of UI faux paus I have encountered in the wild. If you are too lazy, cheap, or busy to read the references above, following these few rules will get a UI most of the way there.

Maintenance First

2008-07-06T10:49:00.000-07:00

"Maintenance typically consumes 40 to 80 percent (average, 60 percent) of software costs. Therefore, it is probably the most important life cycle phase of software."

- Robert L. Glass, Facts and Fallacies of Software Engineering

All desirable qualities of software flow from maintainability. If code is maintainable, it is easy for other developers to read, it is testable, it is easy to optimize, it is easy to add features, it is easy to remove features, and it is easy to fix bugs. This is why writing maintainable code is so important. It maximizes software value, as it allows for leverage of an attribute unique and powerful to software: it is malleable. This allows software to be an asset or investment, because the code is not dependent on its original author(s). This is why unmaintainable software is said to have high amounts of "debt". This cannot be stressed enough to managers. If they view software as an investment, then paying up for good developers makes economic sense. It is cheaper to pay fewer developers to maintain an investment than to pay more developers to write new shanties from scratch when the previous shanty collapses.

Readable

Many engineers and developers think that writing prose or documentation is not their responsibility. This could not be more false with software. Intention revealing code, with succinct, descriptive, variable and method names is critical for maintainability. Rules that apply to natural languages apply to programming languages as well. Large classes and methods are analogous to run-on sentences and paragraphs. Keeping language constructs small makes them easier for the human brain to digest. For a book on writing succinctly and clearly, I have found no better reference than Strunk and White's Element's of Style.

This is part of the discussion where some developers talk about commenting code. I want to make something clear, excessive code commenting does not make code easy to read. In fact excessive commenting is a code smell. The code is probably peppered with comments as an inferior substitute for refactoring the code into smaller more readable classes and methods. Also comments can violate the DRY principle because they tend to get out of sync with the code itself. Comments have two legitimate purposes. First, to explain a piece of code that is non-intuitive, probably due to someone else's poor API. Second, published methods and classes should be commented using baked-in comments so they are available directly within a code-sensing editor.

Testable

I am a huge fan of unit testing. Unit tests enforce separation of concerns. Unit tests are DRY documentation. Unit tests prevent whack-a-mole bug fixing when changing code. Unit tests keep code in a functional state as the code is written. To get the best results from unit testing, use a continuous build that runs your tests, write the tests as early as possible, and make sure the implementer of the code-under-test is the implementer of the tests themselves.

Easy to Optimize

C.A.R. Hoare (or Knuth depending on whom you ask) famously said premature optimization is the root of all evil. This is right up there with Glass's quote on maintenance. Start a large code base with nothing but optimization in mind and you will end up with an unmaintainable, sub-optimal mess when you are done. This is because optimization is best done a posteriori instead of a priori. It is difficult to determine in advance which parts of the code will be called most frequently or which methods will be most expensive. These measurements can be taken via monitoring and profiling after the code is written. An easily changeable (i.e. maintainable) piece of code can then be modified to remove the bottlenecks that are empirically discovered.

Add Features

Adding features can require changes to existing code because the original code was not written to be used in the new manner. Readable, well tested code, that separates concerns, removes the need to shoe-horn new code where it doesn't fit. The old code can be changed with impunity, without worry about unintended side effects, so that the new code fits nicely.

Remove Features

My favorite part of software development is deleting code. Whether it be a feature that is no longer needed, extra operations that proved unnecessary, an uncalled method flagged by static analysis, or even a whole project that was ill-conceived, less code is always better. Less code means less bugs, less to maintain, etc. Don't take this to mean reducing code to the fewest lines possible. Do not sacrifice readability to reduce code, but remove all code that is unnecessary. And do not leave old code as "commented-out". Delete it. If you want it back, use source control. Not using source control? Stop coding.

Fix Bugs

When code is easy to read it is easy to find errors. It is difficult to tell if obfuscated code is doing what it is supposed to, let alone doing something it is not. Code with small classes, and small methods, that reads like natural language, and strongly separates concerns is code in which it is easy to isolate bugs. Unit testing is also useful in this regard. Before modifying any code to fix a bug, write a unit test to expose the bug. Then fix the code to make the test pass. Combined with a continuous build, that bug will not crop up again, which commonly happens when trying to fix other bugs or adding features to highly coupled code.

Here is another quote from the author of my favorite mocking framework, Rhino Mocks:

"If you have an environment that has friction in it, there is an incentive for the developers to subvert the design in order to produce a quick fix or hack a solution to solve a problem. Creating a zero friction environment will produce a system where there is no incentive to corrupt the design, the easiest thing to do is the right thing to do. By reducing the friction in the environment, you increase the system maintainability."

- Oren Eini, creator of RhinoMocks, Zero Friction & Maintainability

I wanted to talk about software maintenance in my first post to make clear the essence of my perspective on software.