Fragmented integration tests – aka the questionable value zone

Fragmented integration tests – aka the questionable value zone

In TDD, different styles of tests can be applied to cover different levels in your code base. Two or three years ago, if you asked me what they were, I would probably have listed them as:

  • Highest granularity – unit tests, quick to run, drive low-level class design
  • Fragmented – integration tests, testing higher-level components and external dependencies (e.g. real SQL database, fake message bus)
  • Whole system at once – slow end-to-end tests, testing for story/feature acceptance at the UI or client API level

However, it has become increasingly clear that fragmented integration tests (somewhere in the middle between unit and full-blown end-to-end integration tests) don’t really provide the same value as their brothers.

They suffer all the disadvantages of both unit and end-to-end tests:

  • Like unit tests, they require setting up mocks/test doubles for collaborating modules
  • Like end-to-end tests, they have a high cost of maintaining external components like databases and integrated third party systems (setting them up, populating them with the right test data, reverting changes for next time etc)
  • Like unit tests, they are brittle and not friendly to refactoring
  • Like end-to-end tests, they are slow to run

… and, over the lifetime of the code base, the only real benefit they provide is slightly faster feedback of problems than end-to-end tests. Is it really worth keeping them?

St9exception with libstdc++

St9exception with libstdc++

Here’s something I encountered today when writing some C++:

try{    throw std::runtime_error("some message");}catch (std::exception e){    std::cout << "error: " << e.what() << std::endl;}

When run, this code will write “error: St9exception”, instead of “some message” to stdout. “St9exception” comes from libstdc++, in which the default value returned by std::exception::what() is the mangled symbol name. The mistake was that I was catching the exception by value, not by reference. (Too much C# perhaps?)

Instead it should have of course been:

try{    throw std::runtime_error("some message");}catch (const std::exception & e){    std::cout << "error: " << e.what() << std::endl;