Toughts about… The Purpose of Testing (Part 3)

There is a lot of space left between class tests (see part 1) and system tests (see part 2).

Class test are very fast and they do not have any special requirements but they fail to find defects in inter class cooperation or the environment.
System tests on the other hand will find any of these defects but are rather slow and require an automatically updated test system with test data in place. Also system tests will tell that there is a defect but – like human testers – are not that good in telling what was the cause for the defect. You may end up with nightly test builds that keep ruining your Monday mornings by telling you that you screwed up the system Friday afternoon.

So there is clearly room for a third type of test here. This type of test should integrate classes as system tests do but – since there are system tests in place – may accept to test only a modular part of the system. I like to call these parts modules or components and their tests module tests.

Module Tests

Just like "unit testing" the ISTQB Glossary of Testing Terms defines "module testing" as the same as "component testing" which is

The testing of individual software components. [After IEEE 610]

In part 1 I suggested to use "class test" instead of "unit test" to define which unit is under test. However, the term "module test" is supposed to be vague since it fills all the space between class tests and system tests. I would define a module test as a test for a separable module of the software in a minimalistic environment. Since this still is quite vague, here an example: if your system consists of several modules deployed on several application servers in a complex network, you try to separate one module deployed on a locally running application server and simulate the missing parts of the system in a way you can verify how your module interacts with them and you can verify the interactions.

Tools

There are some tools to do simulating and verifying. First of all there are Maven plugins to dynamically start a minimal application server before the tests are run (e.g. Jetty, Tomcat). There are also easy ways to use in-memory databases instead of regular ones (e.g. H2) or libraries for mocking a REST service (e.g. WireMock).

Purpose

Even with all that help simulating parts of a system can be quite hard and if you got system tests in place each module test will almost always test something that is already being tested by a system test, so you will need to adjust not only your class and system tests but also the module test when you change a feature. For these reasons you should not write module tests unless you have a module that keeps causing trouble. By no means you should write a complete suite of module tests.

Rapid Feedback

Module tests give you rapid, local feedback on changes. Whenever you see commit message histories like these:

Fixed error in system test.

Fixed error... again.

OK, now this should work. *fingerscrossed*

Final fix, so help us god.

You should consider writing a module test for the misbehaving module and fix that before instead of checking in something that just hopefully will fix the system test (and might as well at some desperate moment even generate more errors).

More Test Cases

Since system tests are slow and resource and execution time-consuming, you may decide not to test all possible test cases but only few critical ones. For module tests are way faster than system tests we don't need to accept that those cases left out are not tested at all. We can simply move them to module tests (and class tests if possible).

Let's consider an interface for a date range that only accepts a certain date format, only future ranges on weekdays, less than three months away. Examples like this generate a tremendous amount of possible test cases. On a test system this might take way to long and therefore we may be content with some basic cases. On separated module that does not persist the date range and does not return HTML feedback but an error/success code will run much faster and therefore we might be able to test a lot more cases.

Document Module Architecture

As class tests document class design and system tests specify system behaviour, module tests document how a module fits into the system architecture. To fulfil this purpose the tests must be quite readable. I suggest using test actors for module tests as well as for system tests. Test actors in module test do not (necessarily) represent human users but other modules interacting with the one under test.

given:
FrontendActor frontend = new FrontendActor()
PersistenceActor persistence = new PersistenceActor()
DateTime firstDay = DateTime.now().plusDays(28)
DateTime lastDay = DateTime.now().plusDays(30)
 
when:
frontend.sendsNewAccommodationRequest(
    customerName: "Customer Name",
    firstDay: firstDay.toString("yyyy-mm-dd"),
    lastDay: lastDay.toString("yyyy-mm-dd"))
 
then:
persistence.gotNewAccommodationRequest(
    customerName: "Customer Name",
    firstDay: firstDay,
    lastDay: lastDay)

The FrontendActor might be just some simple Rest client posting the data to localhost:8080/my/rest/interface which has just been built and deployed on a locally started minimal Tomcat, while PersistenceActor may be a simple JDBC client that fires queries to a just started in-memory H2 database. The code does not tell us on this level, but we can clearly understand how the frontend module, the module(s) under test and the persistence module are supposed to work together.

Isolate Legacy or Externally Maintained Modules

Sometimes you will need to incorporate externally maintained modules or legacy components. These are not written by you and are not well documented and of course they don't come with a sufficient test suite. It might even be closed source so you can not even read the code to understand.

Since you do not understand how the module works and how it is to be used, you start experimenting to find out what you need to know. Module tests can be used to persist and communicate your experimenting. If the module is maintained externally and gets updated you will get failing module tests and can easily adjust your code in the right spots.

In this case writing that module test is not extra work. It just a way to solve a problem permanently instead of temporarily.

Conclusion

Adding module tests to the set of test types enables development teams to gather rapid feedback on critical parts of the overall product. They help to locate defects where system tests only tell us that there is one. As any test, module tests document how its test object works and therefore should be formulated in a human readable way. Also in case of modules that are not maintained by your team, module tests help to isolate problems.

However useful they may be, system test are all that is needed to ensure the product's functionality and value. For most projects class tests are sufficient as a rapid feedback mechanism, allow a lot of test cases and document the architectural intentions of the author of the classes and (summed up) of the module they form. Since writing and maintaining module tests takes time and money we developers must have a good reason to write them.