Unit Tests: taking a step back with 9 simple guidelines

Although Unit Testing should be common knowledge, still a lot of people do not write effective Unit Tests or miss some basic concepts. In this post I'll touch upon some basic guidelines to write effective Unit Tests by using my own rules for writing guidelines. Of course this is subjective and corresponds to my own experience, so feel free to add your own as a comment. If you have no idea how to sell Unit Testing to your manager, you might also want to read one of my previous posts.

#1 Dependency Injection/Inversion of Control MUST be used everywhere

The biggest advantage of DI & testing is the ability to inject classes of your own choice for testing purposes. This makes it much easier to really test a single unit, with all its exceptions, and to mock (see below) certain classes.

Although frameworks such as Spring and Guice make it easier to use DI, you can apply DI perfectly without those frameworks as well by having a dedicated controller/configuration class that simply injects objects into other objects instead of letting the classes instantiate new objects themselves. When testing certain methods, you can easily inject other objects instead.

#2 Tests SHOULD be small and fast

It is a very thin line between actually testing a unit of code and simple integration tests. Certain dependencies, such as DAOs to load data from a database for example, SHOULD be mocked and injected in the actual code so that there is no dependency on a database when simply testing some business logic. This guarantees a fast execution of the test code and the fact that only a simple unit of code is being tested.

Mocking simply means mimicking behaviour of real objects in a controlled way. A very good framework for mocking is for example http://mockito.org/

#3 Unit Tests MUST NOT have side effects

A Unit Test MUST clean up any mess it makes when testing code so that the system is restored to the original state before the test was executed. This makes sure every single test is guaranteed to start from the same deterministic starting position and there are no surprises because a previously executed test has left the system in an inconsistent state.

You MUST use the setup and cleanup methods to handle these kind of things without the need of repeating the same code or polluting the actual test.

#4 Unit Tests MUST NOT depend on each other

The order in which Unit Tests are run MUST NOT be assumed because the order is non-deterministic in standard Unit Test frameworks. Even if you would not comply to item #3, there is no way that you can be sure that a certain test can depend on the result of a previous test, because you have no idea which one will be executed first.

Unit Tests can not depend on each other by design to keep things simple and clean. This is not a bug!

#5 For every bug found, a second Unit Test MUST be implemented 

The whole idea behind Unit Testing is to make sure or prove that code is fit for use. If a bug is found, the code is clearly not fit for use and the code must be fixed. Once the bug is fixed, a new Unit Test MUST be written to proof that the code is now fit for use. This must be repeated for every bug found.

Each new Unit Test SHOULD also be documented with some background information about the bug and with a reference to the bug or issue tracking system when applicable.

#6 Every method implementing business logic MUST at least have one Unit Test

Getters and setters SHOULD NOT be tested since they generally do not contain any business logic, but all other methods MUST have at least one Unit Test implemented to prove that the code is actually fit for use. Whenever a bug is found, item #5 MUST be applied.

Unit Testing user interfaces is much more difficult and MAY be skipped, but at least the controller or presenter (in MVC or MVP respectively) MUST be tested. MVP has some advantages to make Unit Testing easier, but that's a whole other discussion ( see also http://www.martinfowler.com/eaaDev/uiArchs.html)

#7 Unit Tests MUST be kept in the same package as the source code

When Unit Test classes are kept in the same package, it is much easier to test all the methods (except for private methods) of a certain class.

You MUST keep the Unit Tests in another directory though to separate your Unit Tests from your actual application code. This makes releasing much easier.

#8 Unit Tests MUST be time independent

This might sound strange, but when a test is written to run only on Februari 29th or in the morning because your application behaves differently at that specific time for example, your tests are not guaranteed to reflect reality when they are executed on another time. When a job is scheduled to run on a certain time for example, you MUST NOT test the scheduled job itself (the actual code may never be executed), but you MUST test the actual business logic in the job.

For Java specific Unit Testing, be also very carefully when you are dependent on the Date or Calendar classes. There are many problems with those standard implementations (http://parleys.com/#st=5&id=100&sl=1) and pretty much any alternative, such as Joda time (http://joda-time.sourceforge.net/), is better suited, even in normal business logic outside Unit Tests!

#9 Unit Tests MUST NOT load data from a hard-coded location on a filesystem 

All files MUST be put next to the actual test classes and retrieved as a stream through the classloader (ie. in Java: TestClass.class.getClassLoader().getResource AsStream("<package>.<filename>") ).

When files need to be manipulated, you MUST copy them first to a temporary directory (ie. in Java: System.getProperty("java.io.tmpdir")) and manipulate the files over there by injecting the reference to the resource in your actual business logic. You MUST empty the created files in this directory at the end of your test to comply with item #3.

Selling unit testing to managers/executives

A few years ago, I had to defend to an executive the extra cost of implementing unit tests in a project compared to the return it would give. This was during an architecture board meeting and he pretty much caught me off guard, so I had to come up with something really quickly.

Luckily I remembered an article I read a long time ago that compared Japanese and English players in the automotive industry. Overall during the sixties and seventies (maybe even eighties), Japanese built cars were of a higher quality than the cars built in the Western world. The amount of 'broken' cars produced by American or European were incredibly high compared to what Japan produced. Those cars could not be sold as is and had to be taken apart and repaired while they haven't even left the factory, which was very expensive!

To explain unit testing to the executive, I told him the above story. Then I elaborated on the way Toyota solves this by implementing something similar to 'unit tests' to test the quality of the most recent addition in the intermediate product (i.e. unfinished car). They prefer to stop the process and rather get the quality right the first time than to take apart the car at the end of the line for reparation, which is obviously very costly. This saved Toyota a lot of money and made them much more productive at the same time.

By comparing this to a more easy to understand case and showing the financial benefit using unit tests had, the executive was immediately convinced. I didn't even have to give figures, because it made so much sense. He didn't know that Toyota did this and was even surprised to hear it.

I never found back the article, but I did find a book on the Toyota way (http://www.amazon.co.uk/Toyota-Way-Management-Principles-Manufacturer/dp/0071392319/) which elaborates more on the case above and also elaborates on how Toyota works in general, which is also a very interesting read. They might have to re-read it themselves considering the recent events/recalls ;-).

Principle 5 is the one I am referring to and here is a summary of that principle:

Principle 5. Build a culture of stopping to fix problems, to get quality right the first time.

• Quality for the customer drives your value proposition.
• Use all the modern quality assurance methods available.
• Build into your equipment the capability of detecting problems and stopping itself. Develop a visual system to alert team or project leaders that a machine or process needs assistance. Jidoka (machines with human intelligence) is the foundation for “building in” quality.
• Build into your organization support systems to quickly solve problems and put in place countermeasures.
• Build into your culture the philosophy of stopping or slowing down to get quality right the first time to enhance productivity in the long run.

You might argue that Toyota was rather doing integration tests instead of unit tests, but for an executive this is pretty much the same. Hope this helps whenever you have to defend unit testing in your company!

Scala for the busy Java developer

The title of this blog entry is shamelessly inspired on "The busy Java developer's guide for Scala" from Ted Neward. As a busy man myself and having a Java background with interests in Scala, I went looking for sites, books, exercises, tools, etc... to quickly learn the ins and outs of Scala.

There is of course the main site (http://www.scala-lang.org/), with references to some good books, but that simply takes too much time. A good traject to follow and which I followed also, is the one I describe below.

First thing you need is of course a Scala compiler/interpreter. Why bothering installing Scala on your local machine, even if you have already a good IDE installed. It is far easier to start with an online interpreter (http://www.simplyscala.com/), which simply runs in a browser. It does not run on my iPhone (still iOS 3.1.2, I know,...). Maybe it works on a later version -if so, then probably also on an iPad- or on an Android phone or tablet,... Anyone?

Although that same site has some tutorials on it also, they are not sufficient for a real introduction to the Scala languate. For Java developers, there is a much better introduction from Ted Neward on IBM developerWorks. Here is the deeplink with an overview of all the Scala articles from Ted on developerWorks, sorted by date: http://www.ibm.com/developerworks/views/java/libraryview.jsp?site_id=1&contentarea_by=Java&sort_by=Date&sort_order=1&start=1&end=17&topic_by=&product_by=&type_by=All%20Types&show_abstract=true&search_by=scala%20neward&industry_by=

It is very easy to execute the code from the articles inside de interpreter on Simply Scala, just do a copy & paste. When Ted actually implements a runnable application, skip the object definition and execute the body directly in the interpreter. For example in the article "Class Action", he uses the following snippet to exercise the Rational class:

object RunRational extends Application
{
  val r1 = new Rational(1, 3)
  val r2 = new Rational(2, 5)
  val r3 = r1 - r2
  val r4 = r1 + r2
  Console.println("r1 = " + r1)
  Console.println("r2 = " + r2)
  Console.println("r3 = r1 - r2 = " + r3)
  Console.println("r4 = r1 + r2 = " + r4)
}

Just copy anything within the brackets ( {...} ) into the interpreter et voilà... 

Now that you have a good start and if you have some time left, there are a lot of interesting exercises on the internet to enhance your skills (especially the functional part, which is new for Java developers). Project Euler (http://projecteuler.net/) and Rosetta Code (http://rosettacode.org/wiki/Main_Page) contain quite some interesting and challenging exercises which are also used at universities as small projects.

Have fun with Scala! It might be the language of the future on the JVM, time will tell...