.NET, as a mostly-statically typed language, has a lot of really good options for serializing statically-typed objects. Protocol Buffers, MessagePack, JSON, BSON, XML, SOAP, and the BCL’s own proprietary binary serialization are all great for CLR objects, where the fields can be determined at runtime.

However, for data that is tabular in nature, there aren’t so many options. In my past two jobs I’ve had a need to serialize data:

• That is tabular – not necessarily CLR DTOs.
• Where the schema is unknown before it is deserialized – each data set can have totally different columns.
• In a way that is streamable, so entire entire data sets do not have to be buffered in memory at once.
• That can be as large as hundreds of thousands of rows/columns.
• In a reasonably performant manner.
• In a way that could potentially be read by different platforms.
• Into as small a number of bytes as possible.

Protocol Buffers DataReader Extensions for .NET was born out of these needs. It’s powered by Marc Gravell’s excellent Google Protocol Buffers library, protobuf-net, and it packs data faster and smaller than the equivalent DataTable.Save/Write XML:

Usage is very easy. Serializing a data reader to a stream:

DataTable dt = ...;using (Stream stream = File.OpenWrite("C:foo.dat"))using (IDataReader reader = dt.CreateDataReader()){    DataSerializer.Serialize(stream, reader);}

Loading a data table from a stream:

DataTable dt = new DataTable();using (Stream stream = File.OpenRead("C:foo.dat"))using (IDataReader reader = DataSerializer.Deserialize(stream)){    dt.Load(reader);}

It works with IDataReaders, DataTables and DataSets (even nested DataTables). You can download the protobuf-net-data from NuGet, or grab the source from the GitHub project page.

As a Londoner working in the city, I (along with hundreds of thousands of others) catch the tube to work every day. Although you’re typically crammed in a carriage with hundreds of other commuters, the journey is a solitary one, and many passengers turn to their iPhones, iPads and Kindles to pass the time.

For me, it’s a pretty long journey, up to an hour each way to get from Parsons Green to Canary Wharf — a large portion of which is spent deep underground, with no mobile or 3G coverage on my iPhone 4. During this time you can really tell which apps’ data strategies have been properly thought out and designed, and which ones have been hacked together in a hurry.

I’m not going to mention any offending games or apps by name, but these are the sort of things developers should be shot for:

Apps that don’t cache downloaded data for later viewing

Except in special cases (e.g. streaming audio/video, realtime or sensitive content), all downloaded data must be cached locally, and able to be viewed again without an internet connection. Apps that don’t store downloaded content are just crippled web browsers.

Offline App Store reviews

It is unacceptable to nag me to rate your app in the App Store when there is no internet connection. Honestly — don’t ask users to do the impossible.

Repeated connection attempts and repeated error diaglogs

It is unacceptable to repeatedly show error dialogs when there is no internet connection. By all means, try to continuously download updates, but do it silently and in the background. I do not need to be told every 30 seconds that it (still) couldn’t connect.

Locking the UI when checking for updates on startup

When launched, It is unacceptable to block the user in the UI from reading previously-cached data while downloading updated content. By all means, check for updates on start up, but while that is happening, the user must be able to read the previous cached copy, unless they explicitly asked for a refresh from-scratch.

Throwing away cached data then failing to update

It is unacceptable to leave the user with a blank screen because you threw away previously-cached data THEN failed to update. Local data caches should not be cleared until new data has been 100% retrieved successfully and loaded.

Lost form data, please retype

t is unacceptable to ‘lose’ form values and force the user to retype a message when a submit error occurs. Form values must be saved somewhere — either automatically requeued for resubmission, or as a draft that can be edited or cancelled by the user.

Ad priorities

It is unacceptable for an ad-supported app to crash, refuse to start or behave weirdly because it couldn’t connect to the ad server. Are you worried about people using your app, or seeing ads?

Occasionally connected apps

It is unacceptable for any app to refuse to start without an internet connection. Users should always be able to change settings, view and edit previously-cached data and enqueue new requests where possible (to be submitted when a connection is available).

Final thoughts

In general, occasionally-connected computing is a good model for iPhone/Android apps that push/pull data from the Internet. It just needs to be implemented correctly. Like web designers assuming no javascript and using progressive-enhancement to add improved behaviour on top of basic functionality, mobile developers should assume no internet connection is available most of the time — jump at the opportunity to sync when possible, but don’t interrupt the user or assume it won’t fail.

So please, if If you’re a mobile developer, think about us poor Londoners before you deploy your next version!

NHibernate session management and WCF, redux

It’s been nearly a year since I published my method for getting one NHibernate session per WCF operation using Castle’s NHibernate Facility and AutoTx AOP Facility — a method I developed for an application which was having serious session management issues (it was using one session per SQL statement). However, today I ripped it all out.

The method itself still works. There’s nothing wrong with it. As long as you’re only doing one session per WCF operation (or one session per ASP.NET MVC request for that matter), it works great. However, problems arise when you mix other session styles.

For example, our application has a series of long-running background threads for processing jobs on a queue (up to an hour each) — as well as synchronous WCF operations — all requiring NHibernate. In this case, the one-size-fits-all approach using AOP and the container simply couldn’t give us the control we needed over each session’s lifetime, so we removed the Facility from our Windsor container, and instead now do:

• Register the ISessionFactory in the container, singleton scoped.
• Inject ISessionFactory and call ISessionFactory.OpenSession(), ISession.BeginTransaction() and ITransaction.Commit() at outer boundaries of your application (e.g. WCF service implementations), and background jobs executing outside the context of a WCF operation.
• Pass ISession down through your application as a parameter into any methods that access the database. Callees must not hold any reference to it, or try to manage its lifestyle (e.g. closing it, opening their own transactions etc).

(Note: all NHibernate calls are wrapped in a transaction and committed. Even if they are only SELECT statements, it’s still a good practice.)

It has more-or-less the same end effect as AutoTx facility and CallContext-scoped SessionManager, but with the added control of being able to explicitly manage session lifetimes in other situations. It’s a few more lines of code, but it’s also a good example of stop trying to hide your ORM, and just embrace the ISession directly. And a reminder that technical solutions are never infallible, no matter how elegant they might seem.

If someone asked you, what are the building blocks of MVVM, what would you say? Models, Views, and ViewModels, of course. It’s in the name. But really, there are two others:

• Models
• Views
• ViewModels
• Commands
• Events

If Commands and Events aren’t strongly represented in your application, I’d say there is a strong chance it isn’t factored very well.

One problem I see quite often is the Fat ViewModel: a long, bloated class that violates SRP by presenting complex application Commands to the view, and implementing them too.

Symptoms

You can tell you have a Fat ViewModel if any of the following are true:

• Instead of having a separate class for each Command, your ViewModel uses DelegateCommands (aka RelayCommands) to invoke methods on itself.
• Your ViewModel has three properties, two commands, and is 300 lines long.
• Your ViewModel has a large number of services injected into it.
• Your ViewModel tests include assertions for both presentation and application behaviour.
• Testing simple UI behaviour — e.g. a button should be disabled after it has been clicked — requires excessive mocking of dependencies.
• Your ViewModel tests frequently call xyzCommand.Execute().
• Commands cannot easily be re-used by other ViewModels. Your application is not modular.
• ViewModels update their state directly, instead of listening for global application Events.

Instead, if you extract Commands into separate classes, that publish their results via Events, you will benefit by having:

• Skinny and light ViewModels.
• Commands that can be developed and tested in isolation.
• Commands that encapsulate and mask complex logic.
• Separation of presentation logic (ViewModels) from application logic (Commands).
• A catalog of clearly defined and reusable Commands that be composed into new ViewModels.

Whither DelegateCommand?

From the cases I have seen, I have no doubt that DelegateCommands are the primary cause of Fat ViewModels — they encourage developers to implement Commands using local ViewModel methods, which results in poorly-factored code. For this reason, I consider DelegateCommands to be in the same bad code smell category as ServiceLocator: as an anti-pattern, with few legitimate uses. (One is when the Command is entirely self-contained within the ViewModel (i.e. does not collaborate with any other object), the other when genuinely delegating to another object. But in that case, the other object should probably be a Command anyway.)

Can’t decide whether to host your WCF service in IIS or a Windows Service? Consider the additional steps you’ll need to perform, explain, troubleshoot, and write documentation for if you follow the IIS route:

• Ensure IIS is installed.
• Run aspnet_regiis -i to install .NET ISAPI module.
• Run ServiceModelReg –i to install handlers for *.svc file types.
• Creating, starting a new App Pool running as a domain account.
• Set your Application to use the new App Pool instead of DefaultAppPool.

Plus, EITHER: (II6/Server 2003)

• Ensure the .NET 2.0 Web Service Extension is enabled.
• Add the domain account to local IIS_WPG group.
• (If required) cscript.exe adsutil.vbs set W3SVC/AppPools/Enable32BitAppOnWin64 “true”

OR: (IIS7/Server 2008)

• Ensure IIS6 Compatibility components are installed so *.vdproj installers can run.
• Add the domain account to local IIS_IUSRS group.
• (If required) set Enable 32-Bit Applications = true

If you are working in an environment like ours with developers in London, servers in Germany, and the ops team in India, where getting server access is harder than getting an appointment with the pope, I’d recommend sticking with Windows Services unless you really need IIS.

I just got a strange validation bug, trying to buy something on eBay UK:

After about eight attempts (trying different browsers, local/international phone number prefixes etc), I emailed them to report it, and find out maybe if I am doing something wrong. Here is their response:

Hello Richard. My name is Elma from PayPal customer service.

I reviewed your account carefully to check why you are unable to use your card and saw that you were trying to pay a £555.00 GBP transaction with your credit card. It appears that this particular payment cannot be funded by a credit card because of our security model.

I would like to assure you that there is nothing wrong with your credit card or your PayPal account. As part of our commitment to protect you and all our customers, we review all transactions that go through us. This payment got declined because we detected that it might not be safe for you to use your credit card for this transaction. Don’t worry. I assure you that it’s just for this particular transaction. You can still use your credit card for your future transactions.

This is amazing. PayPal are reporting a permanent, this-form-will-never-work-so-don’t-bother-filling-it-out problem to users as an input validation error. And how do users react to validation errors? They double check their details and type them in again. And again. And again, until they’ve gone slightly mad, spending hours scrutinizing every pixel in every digit in their credit card and telephone number for mistakes.

In the end, it wasn’t until I contacted PayPal directly (hint: their feedback form is not exactly easy to find) that I was told my credit card would never work for this particular purchase, and the whole thing was a waste of time. (I wonder how many customers they lost who didn’t bother to fill out a bug report?)

If you are writing software like this, you have BIG problems.

A lot of people have been talking about the Reactive Extensions for .NET, a flash new library from Devlabs which lets you do LINQ to .NET events just like you can with collections. My team at work has started using it pretty heavily, and today I wrote an adapter that allows you to use Rx over events published on the event aggregator in a Prism/Composite WPF application.

Here’s an example of using it in an imaginary chat client, to filter incoming messages from a particular person:

public class ConversationViewModel : IDisposable{    public IContact Friend { get; private set; }    public ObservableCollection<IMessage> Messages { get; private set; }    public ConversationViewModel(IContact friend,         IEventAggregator eventAggregator)    {        // Interested only in messages sent by the person we are talking to.        var messageReceivedEvents = eventAggregator            .GetEvent<MessageReceived>()            .AsObservable()            .Where(msg => msg.Sender.Equals(Friend));        eventSubscription = messageReceivedEvents.Connect();        messageReceivedEvents.Subscribe(msg => Messages.Add(msg));    }        // We will keep receiving messages until the subscription is disposed.    IDisposable eventSubscription;}

It was a good exercise that helped me understand a bit more about how Rx works, and illustrates how easy it is to integrate with callback-based event systems (e.g. your favourite message bus).

You can grab the code here: CompositePresentationEventExtensions.cs

Update Jun 16 2010: Added finalizer to CompositePresentationEventSubscription.

Update Feb 9 2012: Fixed bug found by Ariel Ben Horesh where keepSubscriberReferenceAlive needed to be true.