One question that came up several times at DDD eXchange last week was CQRS: now we understand all the benefits, how do we begin migrating our existing applications towards this sort of architecture?

It’s something we’ve been chipping away at at work recently, and over a short series of posts I’d like to share some of the conventions and patterns we’ve been using to migrate traditional WPF client/WCF server systems towards a CQRS-aware architecture.

Note that WCF isn’t strictly required here — these patterns are equally applicable to most other RPC-based services (e.g. old ASMX web services).

CQRS recap

Command-Query Responsibility Segregation (CQRS) is based around a few key assumptions:

  • All system behaviour is either a command that changes the state of the system, or a query that provides a view of that state (e.g. to display on screen).
  • In most systems, the number of reads (queries) is typically an order of magnitude higher than than the number of writes (commands) — particularly on the web. It is therefore useful to be able to scale each side independently.
  • Commands and queries have fundamentally different needs — commands favour a domain model, consistency and normalization, where reads are faster when highly denormalized e.g. table-per-screen with as little processing as possible in between. They often also portray the same objects differently — commands are typically small, driven by the needs of business transactions, where queries are larger, driven by UX requirements and sometimes including projections, flattening and aggregation.
  • Using the same underlying model and storage mechanism for reads and writes couples the two sides together, and ensures at least one will suffer as a result.

CQRS completely separates (segregates) commands and queries at an architectural level, so each side can be designed and scaled independently of the other.

CQRS and WCF

The best way to to begin refactoring your architecture is to define clear interfaces — contracts — between components. Even if secretly, the components are huge messes on the inside, getting their interfaces (commands and queries) nailed down first sets the tone of the system, and allows you to begin refactoring each component at your discretion, without affecting others.

Each method on our WCF service must be either a command or a query. Let’s start with commands.

Commands

Each command method on your service must:

  • Take a single command argument — a simple DTO/parameter object that encapsulates all the information required to execute the command.
  • Return void — commands do not have return values. Only fault contracts may be used to throw an exception when a command fails.

Here’s an example:

[DataContract]
public class BookTableCommand : ICommand
{
    [DataMember]
    public DateTime TimeAndDay { get; set; }

    [DataMember]
    public int PartySize { get; set; }

    [DataMember]
    public string PartyName { get; set; }

    [DataMember]
    public string ContactPhoneNumber { get; set; }

    [DataMember]
    public string SpecialRequests { get; set; }
}

Commands carry all the information they need for someone to execute them — e.g. a command for booking a restaurant would tell us who is coming, when the booking is for, contact details, and any special requests (e.g. someone’s birthday). Commands like these are a special case of the Parameter Object pattern.

Now we’ve got our command defined, here’s the corresponding method on the WCF service endpoint:

[ServiceContract]
public interface IBookingService
{
    [OperationContract]
    void BookTable(BookTableCommand command);
}

One class per command

Command DTO classes are never re-used outside their single use case. For example, in the situation a customer wishes to change their booking (e.g. change it to a different day, or invite more friends), you would create a whole new ChangeBookingCommand, even though it may have all the same properties as the original BookTableCommand.

Why bother? Why not just create a single, general-purpose booking DTO and use it everywhere? Because:

  1. Commands are more than just the data they carry. The type communicates intent, and its name describes the context under which the command would be sent. This information would be lost with a general-purpose object.
  2. Using the same command DTO for two use cases couples them together. You couldn’t add a parameter for one use case without adding it for the other, for example.

What if I only need one parameter? Do I need a whole command DTO for that?

Say you had a command that only carried a reference to an object — its ID:

[DataContract]
public class CancelBookingCommand : ICommand
{
    [DataMember]
    public Guid BookingReference { get; set; }
}

Is it still worth creating an entire command DTO here? Why not just pass the GUID directly?

Actually, it doesn’t matter how many parameters there are:

  • The intent of the command (in this case, cancelling a restaurant booking) is more important than the data it carries.
  • Having a named command object makes this intent explicit. Not possible just with a GUID argument on your operation contract.
  • Adding another parameter to the command (say, for example, an optional reason for cancelling) would require you to change the signature of the service contract. Ading another property to a command object would not.
  • Command objects are much easier to pass around than a bunch of random variables (as we will see in the next post). For example, you can queue commands on a message bus to be processed later, or dispatch them out to a cluster of machines.

Why not just one overloaded Execute() method?

Instead of having one operation contract per command, why don’t you just use a single overloaded method like this?

[ServiceContract]
public interface IBookingService
{
    [OperationContract]
    void Execute<T>(T command) where T : ICommand;
}

You can but I wouldn’t recommend it. We’re still doing SOA here — a totally-generic contract like this makes it much harder for things like service discovery and other clients to see the endpoint’s capabilities.

Next: Part 2 – Command Handlers

June 15th, 2010 | 20 Comments

Last week, I bought my first new development machine in three years, and it’s a beast.

Early on I decided I wanted a laptop over a big desktop PC. Although laptops are usually more expensive, slower, have smaller screens, and fiddly keyboards, mobility is really important for me — I am planning to be working/travelling around Europe for forseeable future, and lugging a huge desktop everywhere isn’t really feasible. I want to travel light. But also it needs to be powerful enough to run all the heavy developer tools (Visual Studio, VMWare, SQL Server, Photoshop etc) I use every day.

Laptop – i7 MacBook Pro

My last laptop was a Mac (a Powerbook G4), and I was very happy with it — it lasted for five years before I moved to London and saw me through university, two jobs, three flats, and is basically where I learnt to program. So it should not be a surprise that last week, after months of waiting, I finally spec’d out a brand new 15″ i7 MacBook Pro from the Apple online store.

Why a Mac, when I’m primarily a .NET developer? Really, it comes down to these simple reasons:

  1. In my opinion, MacBook Pros are, and always have been the best looking laptops on the market.
  2. They are fast (although still only dual-core).
  3. The build quality of Apple laptops is very high, and it will probably last a long time.
  4. It can happily run Windows.

Going with a MacBook Pro was pretty much a no brainer as far as I’m concerned. But that’s not the end of the story — there’s a lot more to see under the hood.

Solid State Disk (SSD) – 200GB OCZ Vertex LE

It’s long been known that SSDs are fantastic for developers — their ultra-low seek times and small read/write latency can make them several times faster at compiling code than their spinning counterparts. Not to mention it’ll make your PC boot in about ten seconds.

Now SSDs have been available with Macs for some time, but the Toshiba drives used aren’t known for their performance. For a long time I was planning to get an Intel X-25M instead — the long-standing value/performance king of the SSD market. That is, until I saw the absurdly-fast OCZ Vertex 2 Pro.

This thing eats all other SSDs alive. But sadly you will never be able to buy one, because a few months after this review, the entire Vertex 2 Pro product line was cancelled. It seems the cost of the super fast enterprise-grade Sandforce SF-1500 controller was not feasible for OCZ’s desired price range, so they canned it.

However, a small number (only 5,000 units) of SF-1500-based drives were made available to customers as the OCZ Vertex Limited Edition (LE), probably from some early shipment. And according to Anand’s benchmarks, they are actually even faster than the Vertex 2 Pro. So naturally, I had to get one.

The difference is astounding. Here’s a comparison of my XBench results, before and after the upgrade (Seagate Momentus 7200RPM vs OCZ Vertex LE, click for full results):

As you can see, random small (4K) reads and writes is where it really shines — no more waiting for spinning magnetic platters for me!

Memory – 8GB Apple

Historically, Apple’s RAM upgrades have always been notoriously expensive — it’s almost always cheaper to buy/install it yourself. Strangely at the moment, however, Apple’s prices aren’t too bad, so I decided to configure it with 8GB installed — partly for convenience, and partly to ensure my RAM sticks are matched (same specs/manufacturer).

Screen – 15″ Hi Res, Glossy LCD

I have to admit this decision was based entirely on aesthetics — the matte screen is easier to read, particularly in bright sunlight, but that silver bezel is so 2004 :)

The hi res screen is glorious too, but you will want to increase your font size a bit when coding (I always use 14 pt).

Operating System – dual boot Mac OS X/Windows 7 Professional

Although I love Mac OS X, .NET is my game, so I’ll probably be spending most of my time in Windows. This is my first Intel Mac (hey, it’s been a while…) so I’m quite new to the whole Bootcamp/Parallels/VMWare thing, but at this stage VMWare Fusion looks pretty good – being able to run my Bootcamp partition as a VM under Mac OS X seems like a nice solution.

Moshi Palmguard

I got a Moshi Palmguard because of my last Mac – a G4 Powerbook which suffered from ugly aluminum corrosion and pitting on the palm rests. Apparently aluminium pitting is still a problem for Macs today, so I wanted something to help protect it.

My Moshi Palmguard looks fantastic, and was dead easy to stick on — I found it easiest to align it with the cut-out at the bottom of the the track pad (where you open the lid). Just make sure there isn’t any dust or crumbs on the palm rests before sticking it on — you can’t reapply it once it’s attached. Also, I didn’t bother with the trackpad guard — the trackpad on a MacBook Pro is plastic, not metal, so it won’t corrode, and my Powerbook’s still looked fine after five years, so I didn’t see much point.

So that’s it — my proud new developer machine!

Final specs:

  • 15″ MacBook Pro
  • MBP 15″ HR Glossy WS Display (upgraded from standard res)
  • Intel Core i7 M620 dual core 2.66GHz CPU
  • 8GB 1066MHz DDR3 RAM (upgraded from 4GB)
  • 200GB OCZ Vertex LE SSD SF-1500 (upgraded from 500GB 7200RPM Seagate Momentus)
  • SuperDrive 8X DVDRW/CDRW
May 29th, 2010 | 4 Comments

Soft delete is a commonly-used pattern amongst database-driven business applications. In my experience, however, it usually ends up causing more harm than good. Here’s a few reasons why it can fail in bigger applications, and some less-painful alternatives to consider.

Tomato, Tomato

I’ve seen a few different implementations of this pattern in action. First is the standard deleted flag to indicate an item should be ignored:

SELECT * FROM Product WHERE IsDeleted = 0

Another style uses meaningful status codes:

SELECT * FROM Task WHERE Status = 'Pending'

You can even give an item a fixed lifetime that starts and ends at a specific time (it might not have started yet):

SELECT * FROM Policy WHERE GETDATE() BETWEEN StartDate AND EndDate

Highway of death

All of these styles are all flavours of the same concept: instead of pulling dead or infrequently-used items out of active set, you simply mark them and change queries to step over the corpses at runtime.

This is a trade-off: soft delete columns are easy to implement, but incur a cost to query complexity and database performance later down the track.

Complexity

To prevent mixing active and inactive data in results, all queries must be made aware of the soft delete columns so they can explicitly exclude them. It’s like a tax; a mandatory WHERE clause to ensure you don’t return any deleted rows.

This extra WHERE clause is similar to checking return codes in programming languages that don’t throw exceptions (like C). It’s very simple to do, but if you forget to do it in even one place, bugs can creep in very fast. And it is background noise that detracts away from the real intention of the query.

Performance

At first glance you might think evaluating soft delete columns in every query would have a noticeable impact on performance.

However, I’ve found that most RDBMSs are actually pretty good at recognizing soft delete columns (probably because they are so commonly used) and does a good job at optimizing queries that use them. In practice, filtering inactive rows doesn’t cost too much in itself.

Instead, the performance hit comes simply from the volume of data that builds up when you don’t bother clearing old rows. For example, we have a table in a system at work that records an organisations day-to-day tasks: pending, planned, and completed. It has around five million rows in total, but of that, only a very small percentage (2%) are still active and interesting to the application. The rest are all historical; rarely used and kept only to maintain foreign key integrity and for reporting purposes.

Interestingly, the biggest problem we have with this table is not slow read performance but writes. Due to its high use, we index the table heavily to improve query performance. But with the number of rows in the table, it takes so long to update these indexes that the application frequently times out waiting for DML commands to finish.

This table is becoming an increasing concern for us — it represents a major portion of the application, and with around a million new rows being added each year, the performance issues are only going to get worse.

Back to the original problem

The trouble with implementing soft delete via a column is that it simply doesn’t scale well for queries targeting multiple tables — we need a different strategy for larger data models.

Let’s take a step back and examine the reasons why you might want to implement soft deletes in a database. If you think about it, there really are only four categories:

  1. To provide an ‘undelete’ feature.
  2. Auditing.
  3. For soft create.
  4. To keep historical items.

Let’s look at each of these and explore what other options are available.

Soft delete to enable undo

Windows 7 Recycle Bin

Human error is inevitable, so it’s common practice to give users the ability to bring something back if they delete it by accident. But this functionality can be tricky to implement in a RDBMS, so first you need to ask an important question — do you really need it?

There are two styles I have encountered that are achieved via soft delete:

  1. There is an undelete feature available somewhere in the UI, or
  2. Undelete requires running commands directly against the database.

If there is an undo delete button available somewhere for users, then it is an important use case that needs to be factored into your code.

But if you’re just putting soft delete columns on out of habit, and undelete still requires a developer or DBA to run a command against the database to toggle the flags back, then this is a maintenance scenario, not a use case. Implementing it will take time and add significant complexity to your data model, and add very little benefit for end users, so why bother? It’s a pretty clear YAGNI violation — in the rare case you really do need to restore deleted rows, you can just get them from the previous night’s backup.

Otherwise, if there really is a requirement in your application for users to be able to undo deletes, there is already a well-known pattern specifically designed to take care of all your undo-related scenarios.

The memento pattern

Soft delete only supports undoing deletes, but the memento pattern provides a standard means of handling all undo scenarios your application might require.

It works by taking a snapshot of an item just before a change is made, and putting it aside in a separate store, in case a user wants to restore or rollback later. For example, in a job board application, you might have two tables: one transactional for live jobs, and an undo log that stores snapshots of jobs at previous points in time:

If you want to restore one, you simply deserialize it and insert it back in. This is much cleaner than messing up your transactional tables with ghost items, and lets you handle all undo operations together using the same pattern.

Soft delete for auditing

Another common practice I have seen is using soft delete as a means of auditing: to keep a record of when an item was deleted, and who deleted it. Usually additional columns are added to store this information:

As with undo, you should ask a couple of questions before you implement soft delete for auditing reasons:

  • Is there a requirement to log when an item is deleted?
  • Is there a requirement to log any other significant application events?

In the past I have seen developers (myself included) automatically adding delete auditing columns like this as a convention, without questioning why it’s needed (aka cargo cult programming).

Deleting an object is only one event we might be interested in logging. If a rogue user performs some malicious acts in your application, updates could be just as destructive so we should know about those too.

One possible conclusion from this thought is that you simply log all DML operations on the table. You can do this pretty easily with triggers:

-- Log all DELETE operations on the Product table
CREATE TRIGGER tg_Product_Delete ON Product AFTER DELETE
AS
	INSERT INTO [Log]
	(
		[Timestamp],
		[Table],
		Command,
		ID
	)
	SELECT
		GETDATE(),
		'Product',
		'DELETE',
		ProductID
	FROM
		Deleted

CREATE TRIGGER tg_Product_Update ON Product AFTER UPDATE
AS
-- ...etc

Contextual logging

The Hangover

If something goes wrong in the application and we want to retrace the series of steps that led to it, CREATES, UPDATES and DELETES by themselves don’t really explain much of what the user was trying to achieve. Getting useful audit logs from DML statements alone is like trying to figure out what you did last night from just your credit card bill.

It would be more useful if the logs were expressed in the context of the use case, not just the database commands that resulted from it. For example, if you were tracking down a bug, would you rather read this:

[09:30:24] DELETE ProductCategory 142 13 dingwallr

… or this?

[09:30:24] Product 'iPhone 3GS' (#142) was removed from the catalog by user dingwallr. Categories: 'Smart Phones' (#13), 'Apple' (#15).

Logging at the row level simply cannot provide enough context to give a true picture of what the user is doing. Instead it should be done at a higher level where you know the full use-case (e.g. application services), and the logs should be kept out of the transactional database so they can be managed separately (e.g. rolling files for each month).

Soft create

The soft delete pattern can also be extended for item activation — instead of simply creating an item as part of the active set, you create it in an inactive state and flick a switch or set a date for when it should become active.

For example:

-- Get employees who haven't started work yet
SELECT * FROM Employee WHERE GETDATE() &lt; StartDate

This pattern is most commonly seen in publishing systems like blogs and CMSs, but I’ve also seen it used as an important part of an ERP system for scheduling changes to policy before it comes into effect.

Soft delete to retain historical items

Raiders of the Lost Database

Many database-backed business applications are required to keep track of old items for historical purposes — so users can go back and see what the state of the business was six months ago, for example.

(Alternatively, historical data is kept because the developer can’t figure out how to delete something without breaking foreign key constraints, but this really amounts to the same thing.)

We need to keep this data somewhere, but it’s no longer immediately interesting to the application because either:

  • The item explicitly entered a dormant state – e.g. an expired eBay listing or deactivated Windows account, or
  • The item was implicitly dropped from the active set – e.g. my Google Calendar appointments from last week that I will probably never look at again

I haven’t included deleted items here because there are very few cases I can think of in business applications where data is simply deleted (unless it was entered in error) — usually it is just transformed from one state to another. Udi Dahan explains this well in his article Don’t Delete — Just’ Don’t:

Orders aren’t deleted – they’re cancelled. There may also be fees incurred if the order is canceled too late.

Employees aren’t deleted – they’re fired (or possibly retired). A compensation package often needs to be handled.

Jobs aren’t deleted – they’re filled (or their requisition is revoked).

Unless your application is pure CRUD (e.g. data grids), these states most likely represent totally different use cases. For example, in a timesheet application, complete tasks may be used for invoicing purposes, while incomplete tasks comprise your todo list.

Different use cases, different query needs

Each use case has different query requirements, as the information the application is interested in depends on the context. To achieve optimal performance, the database should reflect this — instead of lumping differently-used sets of items together in one huge table with a flag or status code as a discriminator, consider splitting them up into separate tables.

For example, in our job board application, we might store open, expired and filled listings using a table-per-class strategy:

Physically separating job listings by state allows us to optimize them for different use cases — focusing on write performance for active items, and read performance for past ones, with different columns, indexes and levels of (de)normalization for each.

Isn’t this all overkill?

Probably. If you’re already using soft delete and haven’t had any problems then you don’t need to worry — soft delete was a sensible trade-off for your application that hasn’t caused any serious issues so far.

But if you’re anticipating growth, or already encountering scalability problems as dead bodies pile up in your database, you might like to look at alternatives that better satisfy your application’s requirements.

The truth is soft delete is a poor solution for most of the problems it promises to solve. Instead, focus on what you’re actually trying to achieve. Keep everything simple and follow these guidelines:

  • Primary transactional tables should only contain data that is valid and active right now.
  • Do you really need to be able to undo deletes? If so, there are dedicated patterns to handle this.
  • Audit logging at the row level sucks. Do it higher up where you know the full story.
  • If a row doesn’t apply yet, put it in a queue until it does.
  • Physically separate items in different states based on their query usage.

Above all, make sure you’re not gold plating tables with soft delete simply out of habit!

November 20th, 2009 | 26 Comments

A while ago I wrote a post how to lose traction on a personal software project, based on mistakes I have made myself in the past that have slowed down or even completely stopped progress. Today I want to share a tip that has greatly improved my time management since I’ve started doing it, and helped combat many “programmer’s block” moments I am notorious for.

Coming back to work

After you take a break from a coding project for a while, getting back into the groove can be difficult. Stopping for a while can be a great way to gain perspective and maybe reevaluate your goals of what the application should and shouldn’t be, but it can also be really hard starting again where you left off.

If you know you will be leaving for a while, and want to return later, you can try leaving a test failing, or some other obvious problem that needs fixing (like a syntax error), but these only keep you going in the short term. It can be difficult choosing the next big piece of functionality to work on.

Keeping an eye on the big picture

Have you ever been in the situation where, working on an application, you surround yourself with a forest of rich infrastructure code — configuration, data access, logging, UI widgets etc — and then stop and realise that’s all your application is: an empty shell that doesn’t do anything useful yet?

Or alternatively, do you ever find yourself getting lost in detail, giving unfair attention to making one little component perfect while neglecting the other 90% needed to be up and running for the alpha release?

All those little extra “clean up” tasks you want to do

While coding I’m always spotting things I want to clean up like refactorings, missing comments, source code formatting, etc. I want to finish my current task before starting something new (and can’t until I check in my current work anyway), but I do want to do these little clean ups at some stage.

All these problems really boil down to one simple question — what should I work on next?

Patrick Star's to-do list

The programmers’ to-do list

I have found from own personal experience that sitting down to cut code without a well-thought-out plan of attack is asking for trouble. As well as the examples above, I generally skip from task to task, dabbling in things I find interesting, but not gaining much real traction towards a useful application.

Over the past few months, I have discovered that keeping a detailed to-do list is a great tool for combating these problems and staying on track. The premise is very simple: take a high-level block of work — e.g. implementing a single user story — and break it down into all the little programming steps you need to do to get there, no matter how insignificant or obvious.

Here’s a dumb example snippet of a to-do list for a web app I am working on in my spare time (the whole thing runs about four pages total). You can see the ‘high level’ tasks get vaguer towards the end, because I haven’t planned them out yet, and right at the bottom there are unimportant cleanup tasks.

A snippet of my to-do list

The key here is to be fine-grained; you want to see all the little tasks that needs to be done, and then tick them off to see your progress. Even if they are always assumed — like validation — it’s like a calendar, only useful if you know all your appointments are in there. And it doesn’t really matter where your high-level tasks start, as long as together they add up to a program that will be useful to a user.

Defer little tasks

If you think of an easy little ‘clean up’ task you’d like to do, don’t do it now — just write it down instead. Why? Because:

  • If you do it now, it could sidetrack you from your current focus
  • It might be a low priority and a poor use of your time right now
  • Instead of doing it now, it could be an easy way to get back into the groove later

For example, if you’ve got half an hour free before going somewhere, and want to spend it on your application, you don’t want to be starting fresh on some giant new piece of work. Why not spend the minutes crossing off one or two of those little clean-up tasks you’ve been meaning to do?

Planning sessions

Alternatively, if you’re not in the mood to code, you could use the time as a planning session and try to think of the next big task you want to achieve, and all the small steps that make it up. This is just as helpful as cutting coding, and makes you think about the big picture.

To-do list software

I’m a geek, so naturally, I want some flash tool for managing my to-do list. There are a lot to-do list applications on the web like Todoist and Remember the Milk. I tried a few, but eventually just went back to a bulleted, indented Word document because I found it by far the quickest to work with.

July 13th, 2009 | 26 Comments

This Monday, we released the first of several stages in a 6-month upgrade project to a big ERP system. As with any software project, it was a big rush at the end, but particularly so this time because I was lead, and we were doing a lot of architectural improvements (which I will talk about here in a series of articles over the next few weeks).

Anyway, in the race for the deadline, I worked late nights and weekends for sixteen days in a row. The stress was fine, because I was really enjoying the work, but by the end, I found I could hardly hold a conversation with someone about anything other than the project itself. It was as if all my social skills had somehow been sapped — the mind was willing, but the tongue had nothing to talk about.

I’ve never experienced such a thing before, and it was pretty strange. It’s important not to neglect your social life when you’re working overtime to finish a project. It’s very easy to start saying no to parties and spending time with your friends, believing you’re doing a good thing by having early nights and recharging your batteries. My advice to you is DON’T. You need the distraction right now more than ever.

June 25th, 2009 | 9 Comments