Toughts about… The Purpose of Testing (Part 1)

Have you ever seen a test suite that was totally messed up? One where no one on the team could say what a test does and especially why it keeps breaking when you change something that should not affect the test at all (at least judged by its name)? One where there is a high test coverage (according to whatever tool you use) but nobody can tell you where it comes from?

Well, I have seen things like that many times and I think there is a simple single core reason for this: the tests were written without a clear purpose. In this series I will the define the three types of tests and their purpose for development and quality assurance. This first part is about class tests (often called unit tests).

Class Tests

Class tests are the most widely known automated tests. Maybe you are a developer and call them "Unit Test". However, "Unit" is a little too general in my opinion. Which unit do you mean? The ISTQB Glossary of Testing Terms states that "Unit Testing" is the same as "Component Testing", which is

The testing of individual software components. [After IEEE 610]

Not wrong, but not exactly what I (and a lot of developers) really mean. I mean tests for one class – so I call these class tests.

To be precise on this: when running such a test, there should be no activity in any other class. Instances of other classes used by that class must be replaced by mocks, which allow verifying how they were used during the test.1

The test should specify exactly how its class is supposed to work. It calls methods of the class and checks the return value or the way the class interacts with the injected mocks or how the class' state is changed by calling other methods.

Purpose

While writing class tests is daily (though little loved) work to all professional software developers, their purpose is seldom declared. We just feel that they should be there.

Documentation of Intention

First of all, class tests are an executable documentation for the future you and your colleagues. Reading implementation source code can be quite hard, right? We don't know which design goals the original author was trying to meet. There might be some design pattern applied we don't know or do not recognize… Commentary may be helpful but it can also be quite misleading: maybe it was not adjusted to some change in the past or we simply don't understand what the author was trying to say or it is simply not helpful at all:

// setting a here
String a = parameterA + ".";

A class test specifies the intended behaviour of the class and nothing else. Reading test source code therefore is most probably easier (though not necessarily easy) than reading the implementation source code and if it is not failing, it is not outdated. Understanding the test for a class before modifying it keeps us from breaking non obvious side effects.

This purpose of class tests is the most important one to me. However that purpose will only be fulfilled by tests that are quite human readable and findable. There needs to be a clear link between test and code. That's the reason why I think it is necessary to restrict a test to one class and only that one class and name it after that class so we can find it.

Insurance Mechanism for Changes

Ideally, the tests for a class will fail once you change its behaviour in a way not intended by the test author(s) while changes that do not affect the behaviour should affect the test result. So we can use the test in two different ways:

  1. If we actually want to change the behaviour to implement a new feature or fix a bug, we can change the test first to specify the new behaviour so it will tell us when we are finished with our work.
  2. If we do not want to change the behaviour, but just make some optimizations or refactorings, the test will tell us if we accidentally changed the class' behaviour to avoid bugs.

This purpose as well makes it necessary to keep the tested area small and apply a human readable coding style. If a test fails I need to see why it fails otherwise the test will start to be annoying instead of useful.

Test Driven Development/Test First

An extreme variant of intended change of a class' behavior is when the class does nothing at all, since it is not there yet. This method is widely known as Test Driven Development (TDD). It makes the developer think about the purpose of the class and may lead to simpler, more elegant solutions.

Test Coverage

Test coverage is not a purpose! If there is little test coverage on a software we should adjust, we should be very careful since it might be hard to understand the intentions of the original developers. That means if we want to be sure that our changes will do what they should, we need to write the missing tests first.

If the coverage is quite high, the tests might still be confusingly written, so we don't understand the test code. Or the tests make little assertions, so almost any change is accepted by the tests anyway.

Conclusion

Class tests are a simple, easily applied type of test that will keep your test suite from annoying you. At first it might seem unreasonable to write tests that only test one class and not their interaction with each other since there might be errors there as well. However, using a strictly typed language like Java will show us a lot of such errors at compile time and our IDE's make use of that and will tell us while we code. Those errors not covered by that will be found by the other two types of tests I will cover in the next two parts of this series.

Further Reading

A very good and inspiring article about how we should treat our tests is "Test First" by Uncle Bob Martin.

  1. There is only one exception I would make to that rule: if you honestly think you don't need to write a test for the used class. These are often model classes, full of getters and setters without any real logic. []