Showing posts with label instructions. Show all posts
Showing posts with label instructions. Show all posts

March 21, 2011

Unit Tests: taking a step back with 9 simple guidelines


Although Unit Testing should be common knowledge, still a lot of people do not write effective Unit Tests or miss some basic concepts. In this post I'll touch upon some basic guidelines to write effective Unit Tests by using my own rules for writing guidelines. Of course this is subjective and corresponds to my own experience, so feel free to add your own as a comment. If you have no idea how to sell Unit Testing to your manager, you might also want to read one of my previous posts.

#1 Dependency Injection/Inversion of Control MUST be used everywhere

The biggest advantage of DI & testing is the ability to inject classes of your own choice for testing purposes. This makes it much easier to really test a single unit, with all its exceptions, and to mock (see below) certain classes.

Although frameworks such as Spring and Guice make it easier to use DI, you can apply DI perfectly without those frameworks as well by having a dedicated controller/configuration class that simply injects objects into other objects instead of letting the classes instantiate new objects themselves. When testing certain methods, you can easily inject other objects instead.

#2 Tests SHOULD be small and fast

It is a very thin line between actually testing a unit of code and simple integration tests. Certain dependencies, such as DAOs to load data from a database for example, SHOULD be mocked and injected in the actual code so that there is no dependency on a database when simply testing some business logic. This guarantees a fast execution of the test code and the fact that only a simple unit of code is being tested.

Mocking simply means mimicking behaviour of real objects in a controlled way. A very good framework for mocking is for example http://mockito.org/

#3 Unit Tests MUST NOT have side effects

A Unit Test MUST clean up any mess it makes when testing code so that the system is restored to the original state before the test was executed. This makes sure every single test is guaranteed to start from the same deterministic starting position and there are no surprises because a previously executed test has left the system in an inconsistent state.

You MUST use the setup and cleanup methods to handle these kind of things without the need of repeating the same code or polluting the actual test.

#4 Unit Tests MUST NOT depend on each other

The order in which Unit Tests are run MUST NOT be assumed because the order is non-deterministic in standard Unit Test frameworks. Even if you would not comply to item #3, there is no way that you can be sure that a certain test can depend on the result of a previous test, because you have no idea which one will be executed first.

Unit Tests can not depend on each other by design to keep things simple and clean. This is not a bug!

#5 For every bug found, a second Unit Test MUST be implemented 

The whole idea behind Unit Testing is to make sure or prove that code is fit for use. If a bug is found, the code is clearly not fit for use and the code must be fixed. Once the bug is fixed, a new Unit Test MUST be written to proof that the code is now fit for use. This must be repeated for every bug found.

Each new Unit Test SHOULD also be documented with some background information about the bug and with a reference to the bug or issue tracking system when applicable.

#6 Every method implementing business logic MUST at least have one Unit Test

Getters and setters SHOULD NOT be tested since they generally do not contain any business logic, but all other methods MUST have at least one Unit Test implemented to prove that the code is actually fit for use. Whenever a bug is found, item #5 MUST be applied.

Unit Testing user interfaces is much more difficult and MAY be skipped, but at least the controller or presenter (in MVC or MVP respectively) MUST be tested. MVP has some advantages to make Unit Testing easier, but that's a whole other discussion ( see also http://www.martinfowler.com/eaaDev/uiArchs.html)

#7 Unit Tests MUST be kept in the same package as the source code

When Unit Test classes are kept in the same package, it is much easier to test all the methods (except for private methods) of a certain class.

You MUST keep the Unit Tests in another directory though to separate your Unit Tests from your actual application code. This makes releasing much easier.

#8 Unit Tests MUST be time independent

This might sound strange, but when a test is written to run only on Februari 29th or in the morning because your application behaves differently at that specific time for example, your tests are not guaranteed to reflect reality when they are executed on another time. When a job is scheduled to run on a certain time for example, you MUST NOT test the scheduled job itself (the actual code may never be executed), but you MUST test the actual business logic in the job.

For Java specific Unit Testing, be also very carefully when you are dependent on the Date or Calendar classes. There are many problems with those standard implementations (http://parleys.com/#st=5&id=100&sl=1) and pretty much any alternative, such as Joda time (http://joda-time.sourceforge.net/), is better suited, even in normal business logic outside Unit Tests!

#9 Unit Tests MUST NOT load data from a hard-coded location on a filesystem 

All files MUST be put next to the actual test classes and retrieved as a stream through the classloader (ie. in Java: TestClass.class.getClassLoader().getResource AsStream("<package>.<filename>") ).

When files need to be manipulated, you MUST copy them first to a temporary directory (ie. in Java: System.getProperty("java.io.tmpdir")) and manipulate the files over there by injecting the reference to the resource in your actual business logic. You MUST empty the created files in this directory at the end of your test to comply with item #3.




January 05, 2011

A foundation for writing guidelines

I've been several times in a situation where I had to write guidelines for one of our clients . Guidelines, instructions, procedures, protocols, ... some seem more strict than others, but they all suffer from the same problem: how to clearly indicate the do's and don'ts.

A few years ago, I got inspired by what is known as an RFC (Request for Comments) in the internet world. Very roughly speaking, RFCs describes how machines must communicate over the internet. One RFC in particular (RFC 2119), written by Scott Bradner from Harvard University, describes some key words to signify the requirements in a (RFC) document.

Although the RFC clearly states that "they must not be used to impose a particular method they must not be used to try to impose a particular method on implementors where the method is not required for interoperability" (roughly speaking for the internet again), I found them very useful for guidelines in general.

The key words are as follow (excerpt from the originial document):
  1. MUST - This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification.

  2. MUST NOT - This phrase, or the phrase "SHALL NOT", mean that the definition is an absolute prohibition of the specification.

  3. SHOULD - This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.

  4. SHOULD NOT - This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behaviour is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behaviour described with this label.

  5. MAY - This word, or the adjective "OPTIONAL", mean that an item is truly optional.  

Let's look at a simple example of how these key words can be used in practice. Although I don't have grass in my own garden, I found a simple article on "How to mow grass like a pro" with some straight forward instructions on - you guessed it - how to mow a grass like a pro to make it truly look slick. Here is the adapted version that uses the key words to emphasize certain points on what to do and what not to do:

  1. The lawn MUST NOT be wet before mowing grass. Being wet only causes grass to clod up and create mounds of grass through out your lawn. When cutting grass when dry; the clippings spread out evenly, and fall into the lawn and disappears. Of course this depends on how high the grass is before cutting. If grass is exceptionally high, you SHOULD bag the grass and dispose of it.

  2. You MUST NOT cut grass too short. Mowing grass too short allows you to possibly scalp the ground and leave dead spots. As a safe measure, you SHOULD cut grass at about 2 1/2 to 3 inches to be safe. Some even prefer 3 1/2 inches. Depending on how level the ground is; if there are unlevel mounds and drainage trenches, you SHOULD consider cutting as high as possible to avoid scalping.
  3. You SHOULD mow grass before weed eating or trimming. 
  4. This will enable you to make sure you will not have to go over the lawn twice. By weed eating after you mow; those corner spots will stick out like a sore thumb, and you will be able to do a more professional job and not miss anything.

  5. A choice of weed killer can be a number of brands. When you choose a weed killer, you MUST be sure you mix it properly. You MUST read the instructions on the label for mixing it correctly. You MUST NOT spray if the wind is 10 mph or above. You MUST NOT spray around young shrubs and flowers, large trees will handle these weed killers, but take precautions anyway.

  6. You SHOULD spray around your home and barns or storage sheds. You SHOULD spray about 3 inches wide. This will look neat and will not leave a wide dead space that is ugly. You SHOULD NOT spray weed killer in drainage areas on your property, this will only cause erosion eventually. You MAY spray ditches but be advised to only spay the very bottom, you MUST NOT spray the sides.

  7. You MUST use an edger for your side walks and walk ways. You MUST NOT spray these areas or even weed eat them. This will only cause the edge of the grass to get wider, and will not look professional. Edging these areas will have a neat straight look

This looks like overkill, but by using the key words, you make sure that no misinterpretation is possible. Although using the key words does not prevent you of adding arguments or elaborate some parts in the document to clarify things. The key words are there to make sure the requirements are well understood.

All guidelines I write now are superseded by a chapter (eg. "Keywords to indicate requirement levels") describing the above key words. Throughout the whole document, I apply the key words consistently to emphasize requirements in the document. The key words are also in uppercase to even emphasize them more.

It is now much easier to write more to the point and stricter guidelines, without sounding arrogant. By describing the rationale behind the key words in a introductory chapter, they are perceived as a formal part of the document and not as me pointing with a finger.