SqlDropDatabase, SqlCreateDatabase MSBuild tasks

SqlDropDatabase, SqlCreateDatabase MSBuild tasks

When working with SQL, I often find I need to quickly spin up/tear down local developer database instances – for example, setting up a clean environment for integration tests on the build server, or blowing away test data.

To help make this easier, here are a couple of MSBuild tasks I wrote that allow you to drop an existing database:

<SqlDropDatabase ConnectionString="Server=.SQLEXPRESS;Database=AdventureWorks;Integrated Security=SSPI;"                 Database="AdventureWorks" />

… and to create a new (empty) one.

<SqlCreateDatabase ConnectionString="Server=.SQLEXPRESS;Database=AdventureWorks;Integrated Security=SSPI;"                   Database="AdventureWorks" />

It’s also sometimes helpful to be able to parse individual individual keys out of a connection string (e.g. the the database name). This can be very tedious with RegexMatch/RegexReplace, so I wrote a separate MSBuild task to do it:

<SqlParseConnectionString ConnectionString="Server=.SQLEXPRESS;Database=AdventureWorks;Integrated Security=SSPI;">    <Output PropertyName="myDb" TaskParameter="InitialCatalog" />    <Output PropertyName="myServer" TaskParameter="DataSource" />    <Output PropertyName="myTimeout" TaskParameter="ConnectTimeout" /></SqlParseConnectionString><Message Text="Parsed the $(myDb) database on server $(myServer) with timeout = $(myTimeout)." />

Deployment readiness

Deployment readiness

Have you ever seen this development cycle?

  1. Install new build
  2. App doesn’t start, doesn’t log any error why
  3. Go back to source code and add improved logging
  4. Install new build
  5. Read logs
  6. Fix the actual problem (usually some stupid configuration thing)
  7. Repeat steps 1-6 until all bugs are gone

Was it because of your code, or someone else’s? This sort of cycle is time consuming, frustrating, stressful, and makes you look helpless (or like a cowboy) in front of project managers and support staff.

If your app can’t produce logs your infrastructure staff can understand, then your app is not even close to production-readiness. If it doesn’t log clear errors, how can anyone hope to deploy and support it?

Mandatory overloads

Mandatory overloads

Yesterday I read a pretty sensible tweet by Jeremy D. Miller:

(args), you almost always need a Method(Type, args) too.’ width=’550′ height=’292′>

In spirit, I would like to propose another:

When you have a method like Method(IEnumerable<T> items), you should always provide a Method(params T[] items) too.

Workaround for combining a factory method and Lifetime Manager with Unity

Today I encountered a pretty basic scenario that Unity can’t support out of the box: one NHibernate ISession per HttpContext.

The problem is that ISessions are provided by a factory method NHibernateHelper.OpenSession(), and I need to store them in a custom HttpContextLifetimeManager. Unity won’t let me use the two together because no RegisterFactory() overload allows you to specify a LifetimeManager.

Rupert Benbrook has posted a solution to this problem that overrides part of the Unity build-up pipeline (and even goes as far as emitting op codes directly). For simple applications, however, a plain extension method might suffice:

public static IStaticFactoryConfiguration RegisterFactory<T>(    this StaticFactoryExtension extension,     FactoryDelegate factoryMethod, LifetimeManager lifetimeManager){    FactoryDelegate wrappedFactoryMethod = delegate    {        LifetimeManager manager = lifetimeManager;        FactoryDelegate method = factoryMethod;        IUnityContainer container = extension.Container;        object o = manager.GetValue();        if (o == null)        {            o = factoryMethod(container);            manager.SetValue(o);        }        return o;    };    return extension.RegisterFactory<T>(wrappedFactoryMethod);}

This simply wraps the real factory method in a closure that checks a LifetimeFactory first. I can then do:

container.Configure<StaticFactoryExtension>()    .RegisterFactory<ISession>((c) => NHibernateHelper.OpenSession(),     new HttpContextLifetimeManager<ISession>());

This seems to work pretty well.

Entity validation and LINQ: Using yield return to optimize IsValid over a list of broken rules

Entity validation and LINQ: Using yield return to optimize IsValid over a list of broken rules

A common pattern for checking an entity is valid involves testing a number of rules on it. After all tests have been performed, a list of broken rules is returned.

Consider this example for validating instances of a simple Customer class:

class CustomerValidator{    public IEnumerable<RuleViolation> GetAllRuleViolations(Customer c)    {        IList<RuleViolation> ruleViolations = new List<RuleViolation>();        if (String.IsNullOrEmpty(c.FirstName))        {            ruleViolations.Add(new RuleViolation("FirstName",                 "First name cannot be empty."));        }        if (String.IsNullOrEmpty(c.LastName))        {            ruleViolations.Add(new RuleViolation("LastName",                 "Last name cannot be empty."));        }        if (!Regex.IsMatch(c.PhoneNumber, @"[d ()+-]+"))        {            ruleViolations.Add(new RuleViolation("PhoneNumber",                 "Invalid phone number."));        }        return ruleViolations;    }}

Quite often though, we don’t care about the full list of broken rules — we only care if the object is valid or not. Instead of…

IEnumerable<RuleViolation> brokenRules =     customerValidator.GetAllRuleViolations(customer);if (brokenRules.Count() > 0)    // do stuff</pre>...we would rather have:</p><pre lang="csharp">if (!customerValidator.IsValid(customer))    // do stuff

So what’s the difference between checking if an entity is valid and getting a detailed list of validation errors?

For starters, the only way of finding out if an entity is valid is by testing all the rules against it. Let’s assume this is a reasonably intensive operation — if you have a lot of rules, or need to check things with external systems (checking a username doesn’t already exist in the database, for example).

If all we’re doing is checking if the entity is valid, we want to do as little work as possible. This means stopping as soon as we hit a broken rule.

The easiest way to do this is with the yield return keyword. Yield return is kind of strange — it lets you iterate over objects as they are returned from a method. This is used for evaluating LINQ expression trees. Instead of filtering and reducing a collection one criteria at a time — e.g. testing 1000 objects, then re-testing the 500 objects that passed, etc — it tests each object against all the criteria at once.

In this case, we simply want to return as soon as a broken rule is encountered.

class CustomerValidator{    public IEnumerable<RuleViolation> GetAllRuleViolations(Customer c)    {        if (String.IsNullOrEmpty(c.FirstName))        {            yield return new RuleViolation("FirstName",                 "First name cannot be empty.");        }                if (String.IsNullOrEmpty(c.LastName))        {            yield return new RuleViolation("LastName",                 "Last name cannot be empty.");        }        if (!Regex.IsMatch(c.PhoneNumber, @"[d ()+-]+"))        {            yield return new RuleViolation("PhoneNumber",                 "Invalid phone number.");        }    }}

See that? The method is still defined as returning a collection, but it has three return statements with a single object each. With this, we can use a little bit of LINQ to break out of the method early:

    public bool IsValid(Customer c)    {        // are there any rule violations?        return !GetAllRuleViolations(c).Any();    }

I whipped up a little test app to prove this — IsValid vs GetAllRuleViolations.

CustomerValidator validator = new CustomerValidator();// An invalid customerCustomer customer = new Customer(){    FirstName = "",    LastName = "",    PhoneNumber = "asdsd"};// do as little work as possible to work out if the entity is valid or notConsole.WriteLine("IsValid = ");Console.WriteLine(validator.IsValid(customer));// check everything for a full reportConsole.WriteLine("GetAllRuleViolations().Count =");Console.WriteLine(validator.GetAllRuleViolations(customer).Count());

Here’s what it produces. Note that IsValid returns immediately after the first rule fails.

IsValid =        Testing first nameTrueGetAllRuleViolations().Count =        Testing first name        Testing last name        Testing phone number3

Are your applications ‘legacy code’ before they even hit production?

Are your applications ‘legacy code’ before they even hit production?

When a business has custom software developed, it expects that software to last for a long time. To do this, software must A) continue operating — by adapting to changing technology needs, and B) continue to be useful — by adapting to changing business needs.

Businesses’ needs change all the time, and all but the most trivial of software will require modification beyond its original state at some point in its life, in order to to remain useful. This is sometimes known as brownfield development (as opposed to greenfield), and from my experience, accounts for a good two thirds to three quarters of all enterprise software development.

Anyway, time for a story. A company has a flash new business intelligence system developed (in .NET) to replace an aging mainframe system. Everything goes more-or-less to plan, and the system is delivered complete and on time. It’s robust and works very well — everyone is satisfied.

Now, six months have passed since the original go-live date, and the owners have a list of new new features they want added, plus a few tweaks to existing functionality.

Development begins again. After a few test releases, however, new bugs start to appear in old features that weren’t even part of phase two. Delivery dates slip, and time estimates are thrown out the window as search-and-destroy-style bug fixing takes over. Developers scramble to patch things up, guided only by the most recent round of user testing. Even the simplest of changes take weeks of development, and endless manual testing iterations to get right. The development team looks incompetent, testers are fed up, and stakeholders are left scratching their heads wondering how you could possibly break something that was working fine before. What happened?

The illusion of robustness

The robustness of many systems like this is an illusion — a temporary spike in value that’s only reflective of the fact it has been tested, and is known to work in its current state. Because everything looks fine on the surface, the quality of source code is assumed to be fine as well.

From my experience, a lot of business source code is structured like a house of cards. The software looks good from the outside and works perfectly fine, but if you want to modify or replace a central component — i.e. a card on which other cards depend — you pretty much have to re-build and re-test everything above it from scratch.

Such systems are written as legacy code from the very first day. By legacy, I mean code that is difficult to change without introducing bugs. This is most commonly caused by:

  • Low cohesion – classes and methods with several, unrelated responsibilities
  • High coupling – classes with tight dependencies on each other
  • Lack of unit tests

Low cohesion and high coupling are typical symptoms of code written under time-strapped, feature-by-feature development, where the first version that works is the final version that goes to production.

Once it’s up and running, there’s little motivation for refactoring or improving the design, no matter how ugly the code is — especially if it will require manual re-testing again. This is the value spike trap!

The tragedy of legacy code

Back to the story. If a developer on the project realises the hole they’ve fallen into, they can’t talk about it. This late in the process, such a revelation could only be taken in one of two ways — as an excuse, blaming the original designers of the software for mistakes made today — or as a very serious admission of incompetence in the development team from day one.

The other developers can’t identify any clear problems with the code, and simply assume that all brownfield development must be painful. It’s the classic story — when granted the opportunity to work on a greenfield project, you’re much happier ó unconstrained by years of cruft, you’re free to write clean, good code. Right? But if you don’t understand what was wrong with the last project you worked on, you’ll be doomed to repeat all of its mistakes. Even with the best of intentions, new legacy code is written, and without knowing it, you’ve created another maintenance nightmare just like the one before it.

Clearly, this is not a good place to be in — an endless cycle of hard-to-maintain software. So how can we break it?

Breaking the cycle

I believe that Test Driven Development (TDD) is the only way to keep your software easy-to-modify over the long-term. Why? Because:

  1. You can’t write unit tests for tightly-coupled, low cohesive code. TDD forces you to break dependencies and improve your design, simply in order to get your code into a test harness.
  2. It establishes a base line of confidence — a quantifiable percentage of code that is known to work. You can then rely on this to assert that new changes haven’t broken any existing functionality. This raises the value of code between ‘human tested’ spikes, and will catch a lot of problems before anyone outside the development team ever sees them.

TDD gives you confidence in your code by proving that it works as intended. But object-oriented languages give you a lot freedom, and if you’re not careful, your code can end up as a huge unreadable mess.

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” – Martin Fowler

So how do you write code that humans can understand? Enter TDD’s partner in crime, Domain Driven Design (DDD).

DDD establishes a very clear model of the domain (the subject area that the software was written to address), and puts it above all technological concerns that can put barriers of obfuscation between the business concepts and the code used to implement them (like the limitations of a relational database schema). The end result is very expressive entities, repositories and service classes that are based on real life concepts and can be easily explained to non-technical business stakeholders.

Plus, DDD provides a clear set rules around all the fiddly stuff like relationships and responsibilities between entities. In short, DDD is simply object-oriented code done right.

Putting it to practice

So how do you get on to a point where you can use all this? There’s a lot to learn — an entire universe of information on top of your traditional computer science/software development education. I’ve only scratched the surface here in trying to explain why I think it’s important, but if you want to learn more:

  • Read books. How many books on software development have you read this year? Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans is pretty much the DDD bible, and, just like GoF, should be mandatory reading for any serious developer. If you’re stuck in legacy code hell right now though, have a read of Michael Feathers’ Working Effectively with Legacy Code. It’s all about breaking dependencies and beating existing code into test harnesses so you can get on with improving it without fear of unintended consequences.
  • Read blogs. TDD/DDD is a hot subject right now, and blogs are great for staying on top of all the latest tools and methods. They’re also a good place for learning about TDD/DDD’s support concepts like dependency injection, inversion of control, mocking, object/relational mapping, etc.
  • Join mailing lists — DomainDrivenDesign and ALT.NET (if you’re that way inclined) are two good ones with a reasonably high volume of great intellectual discussion. You can gain a lot of insight reading other people’s real-life problems, and watching the discussion evolve as potential solutions are debated.
  • Practise. Grab an ORM, a unit test framework/runner, an IoC container and start playing around.

I believe the only way to make software truly maintainable long-term (without becoming legacy code) is to use TDD and DDD. Together, they’re all about building confidence — confidence that you can make changes to a system without introducing any bugs. Confidence that other people can understand your code and the concepts it represents. And most importantly, confidence that you’re learning, and aren’t making all the same mistakes every time you start a new project.

Troubleshooting Windows module dependencies

Troubleshooting Windows module dependencies

I have just got a new computer at work, and over the past week I have been installing all the software that I like to use. One tool I rely on is IISAdmin, a program that sits in the system tray and allows you to run multiple IIS websites under non-server editions of Windows. Unfortunately, when I tried to install it under Window Vista, it failed with the message “Error 1904. Module C:Program Filesiisadminztray.ocx failed to register. HRESULT -2147024770. Contact your support personnel.”

IISAdmin setup error 1904

Attempting to manually register the module with RegSvr32 didn’t work either:

The module "ztray.ocx" failed to load.

I couldn’t find anything searching for either of the error messages, so I did some digging on the module itself. It turns out that ztray.ocx is an old (1997) ActiveX control that allows programs to add an icon to the system tray.

I eventually found Dependency Walker, a tool that scans Windows modules for dependencies. I opened up ztray.ocx to see if it was missing anything.

Viewing ztray.ocx with Dependency Walker

As you can see, ztray.ocx makes calls to a module called msvbvm50.dll, part of the Visual Basic 5.0 run-time. This package is present in Windows XP, but appears to have been dropped from Vista. Luckily it is still available as a free download from Microsoft. Installing it solved the dependency problem, and I was able to install IISAdmin successfully.