As programmers, we often hear that test code should be a first-class citizen of the project, meaning that it is developed to the same standards, using the same patterns & practices as your production code. Treating your test code this way should make it easier to use and maintain in the long run. So why does test code get so little attention from developers?
I can’t speak for everyone, but it seems to me that one of the common pain points of software testing is establishing the context in which the tests are run. Before you can verify that the system behaves a certain way in a given set of circumstances, you have to create that set of circumstances. More often than not, setting up the context for a test requires more code than the test itself, and sometimes more than the code being tested. This testing code tends to be somewhat dull, and not very rewarding for the developer. It’s grunt-work, and we don’t like it.
I’ve worked in many projects with radically different approaches to building test contexts, and it seems to me that the more “clever” we try to be, the more it comes back to hurt us later on. Inheritance-based approaches give us a high degree of code reuse, but at the expense of clarity. From any given test, it can be hard to understand the context or “world” in which the test takes place because so much of it is hidden in multiple layers of base classes.
The more successful testing approaches I’ve used have all had one thing in common… simplicity. With that in mind, I’ve been trying to find an approach that provides a high degree of code reuse, but makes it easy to see and understand the context. My current approach attacks these two problems via separate, but complimentary, techniques.
Before getting into the details, I’d like to review the evolution of a typical testing framework. In most projects, the tests start off simply enough, with each test being responsible for its own context. As redundancies start to emerge, they are extracted out to methods which get shared by multiple tests (e.g. CreateAccount). When it becomes clear that some of these methods are needed by tests in other classes they are typically pushed into either a base class (e.g. EntityTestBase), or some kind of helper class (e.g. EntityTestHelper). Both of these solutions have a tendency to degenerate into a unmaintainable “God classes”, and so the Create methods eventually get divided off into their own classes (e.g. AccountTestHelper), or exposed as part of the test class for each individual entity (e.g. AccountTests.CreateAccount). These factory methods begin to sprout a lot of arguments to allow them to be used by a multitude of tests, each with slightly different requirements (e.g. CreateAccount(bool includeAddress, bool includeOrders, bool includeLineItems)). As the number of these arguments increases, they may be combined together as properties of some “options” class to make them easier to deal with (e.g. CreateAccount(CreateAccountOptions options)). This last step is as sophisticated as the test code typically gets out there in the wild, and is representative of the majority of the testing code I’ve seen.
For most cases, this is perfectly adequate, and is the approach I’ve seen used on many projects. The static FooTests.CreateAccount method is available for use by any test that happens to need an Account, and the CreateAccountOptions class makes it obvious what choices are available. Nesting options classes allows us to specify properties of children, grandchildren, etc.
What we have at this point is a method, CreateFoo, and its associated parameters, CreateFooOptions. There is a standard software design pattern that fits this functionality almost perfectly, the “Command” pattern. A typical command implementation consists of a set of parameters, and code which uses them to call a method which is usually defined elsewhere. Less “pure” implementations sometimes include the code for the method directly within the command itself, and it is this approach that I will use here.
The Command pattern can also support multi-level undo, which is particularly useful when it comes to cleaning up after integration tests. In most cases, you can simply roll back database transactions to cover your tracks. Sometimes you can’t, though, such as when entities were created by calling remote services that do not support the concept of transactions, or when the system under test creates files. In these cases, having an Undo method which can remember and delete its own test data will be very useful. Undo isn’t always needed, but it’s nice to have around sometimes.
Here is an example command for creating Address entities. I won’t go into the details of the CreateCommand base class here, but the sample project and supporting classes are available on GitHub here.
- public class CreateAddressCommand : CreateCommand<Address>
- {
- private static int _id = 1;
- public int AddressId { get; set; }
- public string Address1 { get; set; }
- public string Address2 { get; set; }
- public string City { get; set; }
- public string State { get; set; }
- public string Zip { get; set; }
- public override void Execute()
- {
- if (Result == null)
- {
- Result = new Address
- {
- AddressId = AddressId,
- Address1 = Address1,
- Address2 = Address2,
- City = City,
- State = State,
- Zip = Zip
- };
- // Code here to write Address to database
- // (e.g. AddressRepository.Add(Address);)
- }
- }
- }
To use CreateAddressCommand, a test would create a new instance of the command, fill in the properties the resulting Address object should have, execute the command, and extract the result. Instead of creating an Address, We’ve just created a command to create the Address. So far, this command doesn’t really do anything we couldn’t have done ourselves. In fact, all this command has done is to add a level of abstraction, and contrary to popular wisdom, it hasn’t solved anything. Stay with me, because we’re not done with it yet.
Next, we’ll build a factory to create pre-defined instances of this command. By adding simple static factory methods to the CreateAddress command, we can define any number of pre-fabricated commands of various descriptions. You could define as many of these methods for as many scenarios as you like. Just make sure to give them descriptive names. For instance, if you were building a system on top of the venerable Northwind database, you might define a CreateCustomerCommand, with a factory method called “AlfredsFutterkiste” which would return you a pre-defined Customer object with example orders, line items, and address information that more or less duplicates a subset of the real database data. Here, I’ve defined a factory methods that returns a “valid” Address by filling in the fields so as to pass object validation.
- public static CreateAddressCommand Valid()
- {
- var result = new CreateAddressCommand
- {
- AddressId = _id++,
- Address1 = GetRandom.String(1, 30),
- Address2 = GetRandom.String(1, 30),
- City = GetRandom.String(1, 20),
- State = GetRandom.String(2, 2),
- Zip = GetRandom.String(10, 10),
- };
- return result;
- }
These factory commands could return a single object, or a customer complete with address, order history, and billing information. Each command can leverage other commands to create a usable test context. This is particularly valuable in an Agile development environment in which the definition of “Valid” may change many times as the project matures. By centralizing the code which creates objects in various states, we should be better able to adapt to changing rules by updating a single factory method instead of a lot of individual unit tests.
Address is a pretty simple “Leaf” object. It doesn’t have any children, and is completely unaware of its own parents. Lets examine a more complex example. This is what a CreateCustomerCommand might look like.
- public class CreateCustomerCommand : CreateCommand<Customer>
- {
- private static int _id = 1;
- public CreateAddressCommand CreateAddressCommand { get; set; }
- public List<CreateOrderCommand> CreateOrderCommands { get; set; }
- public int CustomerId { get; set; }
- public string FirstName { get; set; }
- public string LastName { get; set; }
- public override void Execute()
- {
- if (Result == null)
- {
- Result = new Customer
- {
- Address = GetResult(CreateAddressCommand),
- CustomerId = CustomerId,
- FirstName = FirstName,
- LastName = LastName,
- Orders = GetResults(CreateOrderCommands),
- };
- // Code here to write Customer to database
- // (e.g. CustomerRepository.Add(Customer);)
- }
- }
- public override void Undo()
- {
- base.Undo();
- // Code here to erase Customer from database
- // (e.g. CustomerRepository.Erase(Customer.CustomerId);)
- Result = null;
- }
- public static CreateCustomerCommand New()
- {
- return new CreateCustomerCommand();
- }
- public static CreateCustomerCommand None()
- {
- return null;
- }
- public static CreateCustomerCommand NoOrders()
- {
- var result = New();
- result.CreateAddressCommand = CreateAddressCommand.Valid();
- return result;
- }
- public static CreateCustomerCommand Valid()
- {
- var result = New();
- result.CustomerId = _id++;
- result.CreateAddressCommand = CreateAddressCommand.Valid();
- result.CreateOrderCommands = new List<CreateOrderCommand>
- {
- CreateOrderCommand.Valid(),
- };
- return result;
- }
- }
There are a few new items to discuss here. The CreateAddressCommand and the CreateOrderCommands collection allow tests to describe various child entities of the parent. The New and None factory methods are added by convention for the sake of clarity and consistency. Each command can expose as many static factory methods as needed to return command instances in a variety of pre-determined configurations such as “New”, “Valid”, or even “WithOpenOrders”. Factory methods can be defined for any situation which would see enough reuse to justify it.
Notice again that the command doesn’t actually create anything until Execute is called. The command hierarchy represents the intent to create objects, and not the objects themselves. As a result, tests have the chance to further manipulate the command hierarchy and make changes before executing it. Also, up to this point, all of the work has happened quickly, and in memory. For unit tests, this is not as much of a concern, but for integration tests this could result in significant savings by allowing tests to “prune” unneeded command branches before they are executed.
This chance to modify the plan also allows us to easily create multiple similar contexts by starting at a common starting point (such as “Valid”), and adding, removing, or changing the commands that describe it. The addition of a few more methods can make this customization even simpler. Here are some instance methods that manipulate or modify an existing command prior to execution.
- public CreateCustomerCommand WithAddress(Address value)
- {
- CreateAddressCommand = new CreateAddressCommand {Result = value};
- return this;
- }
- public CreateCustomerCommand WithAddress(CreateAddressCommand command)
- {
- CreateAddressCommand = command;
- return this;
- }
- public CreateCustomerCommand WithCustomerId(int value)
- {
- CustomerId = value;
- return this;
- }
- public CreateCustomerCommand WithFirstName(string value)
- {
- FirstName = value;
- return this;
- }
- public CreateCustomerCommand WithLastName(string value)
- {
- LastName = value;
- return this;
- }
- public CreateCustomerCommand WithOrders(IEnumerable<Order> orders)
- {
- CreateOrderCommands = new List<CreateOrderCommand>(orders.Count());
- foreach (var order in orders)
- {
- CreateOrderCommands.Add(new CreateOrderCommand {Result = order});
- }
- return this;
- }
- public CreateCustomerCommand WithOrders(IEnumerable<CreateOrderCommand> commands)
- {
- CreateOrderCommands = new List<CreateOrderCommand>(commands);
- return this;
- }
These commands make it easy to describe the desired object hierarchy in simple terms like CreateCustomerCommand.Valid().WithAddress(CreateAddressCommand.None()). It’s not English, but it is expressive and clear. I want to start with a valid customer, but make sure the address isn’t filled in. Again, you can define as many of these helper methods as you want.
The Command pattern takes care of the reuse problem, but hasn’t done a lot to increase the readability of our tests. Fortunately, that problem is even easier to slave. There are several BDD-style frameworks out there that seek to remedy the readability problem by enforcing the standard Given/When/Then structure. However, none of these frameworks do so in a way that I’ve been entirely comfortable with. I wanted something simpler, more streamlined, and something that uses the C# language in ways that it was originally designed to be used rather than forcing a fluent syntax where it doesn’t fit. I maintain that if you ever find yourself defining a class called “It”, you’ve probably made a wrong turn somewhere.
My current solution to the readability problem is to mercilessly apply the concept of “self-documenting code”. My test methods consist of nothing but calls to other methods, whose names all begin with “Given”, “When”, or “Then”. The resulting tests look similar to this:
- [TestMethod]
- public void CreateAddress_returns_a_valid_Address_by_default()
- {
- Given_a_valid_Address();
- When_IsValid_is_called();
- Then_IsValid_is_true();
- }
The Given_a_valid_Address method encapsulates the creation and execution of the CreateAddressCommand, and saves the result to an appropriate backing variable. When_IsValid_is_called exercises the code we want to test, in this case the Address validation, and assigns the result to another backing variable. Finally, Then_IsValid_is_true performs the actual testing of the results of the first two methods.
There’s not really any more code to show here. It’s just a simple idea. Factor each step out to its own method with an intelligent name. You can move these methods to a base class if you want to share them across multiple test classes, but you won’t lose your way now trying to remember exactly what the context is because it’s explicitly listed out in the beginning of each test.
The sample project and supporting classes are available on GitHub here.
MelGrubb.ToBlog() 