The Testing Anti-Patterns Drafts Vol. 1 (draft2)

George and I both consider ourselves fortunate enough to work as members of teams where TDD is practiced by default.

There are two reasons beyond the obvious as to why we find it relatively harder to code in a non TDD approach. Test Driving our code enhances our vision on how to design/model/instrument the application's universe. Also, starting with a test means work begins and ends with coding, not meetings, discussions or modelling our vision in pictures bound to be proven unrealistic when the first coding bottlenecks arise.

This is not to say that discussions or modelling are intrinsically bad things, a good brisk white board discussion can really help – but don't ever forget that this is a relatively abstract activity.

Tests have a much closer relationship with your code that allows you to discover and document the behaviour of your system in a much more detailed manner. In fact it is the close relationship between tests and code that can lead to problems. This relationship must be kept in balance if the process is to be successful.

Production code is the bread winner. This is where your business value lies. However, due to the nature of the TDD process, the production code is also somewhat dim-witted; it doesn't really have a clear vision or motivation in life. The test code is there to explain to the code what is expected of it and to highlight any mistakes the code makes by identifying in a precise manner why the code doesn't quite do the right thing and to point out how to head off in the correct direction. In some respects this could be likened to the relationship between a boxer and his trainer. Ultimately, it is the boxer who has to go into the ring and win the fight, but it is the trainer who gets the boxer into the zone mentally and physically.

Another equally important role of the test code is to document the expected behaviour of the production system. This is fantastic! Suddenly, the technical documentation is no longer a static and dusty tome on a shelf, rather an ever accurate and clear guide to what the code really does. If your code is not fit for purpose, this should be made glaringly obvious in your tests, and by changing your documentation you should be able to change the behaviour of your code to make it do the right thing in the right way.

However, we don't live in a perfect world and testing properly is not easy, despite the presence of opposable thumbs and shiny new Macs. Once you get past the simplistic examples and into the real world you find that it is difficult to write effective tests, and often the tests are much harder to craft than the actual code. In many ways, this should not come as a surprise. When you write a test, you are attempting to satisfy a number of aspects of the system: quality, design, documentation. And all of this expressed though the medium of a software language!

So, what makes our tests go bad? There seem to be a number of patterns starting to emerge, but to list them all here is going to take too long – I think George and I are going to have to break this out (hopefully with lots of assistance from our friends ;-)). In my mind the coarse categories of TDD anti patterns are:

1. Driving the code the wrong way.
2. Hiding deficiencies in the code.
3. General test smells
   

Test Driving The Wrong Way

This is the situation where your code starts to bend towards the test code to meet requirements of the testing framework rather than to meet the needs of the application. Examples of this include:

• Adding calls to your production data layer code to allow setup/teardown/query operations which are not required by the application.


• Providing access to properties which should not be made visible normally (this is what George is getting at with Design Pervasive Testing)


• Forcing the use of IOC where in fact it makes more sense to create objects internally or access static methods.
  

Clearly there is a sliding scale of smelliness here. Overuse of IOC is not really such a bad thing – the code will still function as required. It just leaves you thinking that there has to be a better way.

Adding a deleteAllClients() method to your production code is a big deal though!

Sometimes, this can be sign that the test framework that you are using to test your application is not suitable, as switching to a different technology can help get around the smell.

For example - the use of DbUnit may make it easier to put your database into the correct state for a functional test and remove the need for the dodgy setup code in your data layer. Newer mocking libraries like JMockit can remove the absolute requirement for IOC based design patterns by allowing you to hook the creation of new objects and accessing static methods.

Hiding deficiencies in the code

This is where we seem to be suffering the most. It is very easy to hide problems in the code base instead of actually driving them out with your tests. Examples of this phenomenon:

• Hidden behaviour. The production code performs a variety of complex activities in a given situation. However this is not clear because the test has buried the behaviour in deep in a number of (often obscurely named) helper functions. This is often a sign that you have too many complex relationships between your objects, or that the conversations the component has with its collaborators is overly chatty.


• Supporting actors stealing the show. The test is so polluted with setup code that you can't actually make out what the test is trying to show you. This can often be a sign that the step is too complex.
  

General Test Smells

• Badly named tests. This is especially bad when you consider the need to document the code through the medium of tests.


• Inappropriate use of stubs. For example, stubbing a simple data type.


• Etc.
  

In summary, I think we really need to care about the quality or our test code and learn to treat it quite differently to the production code, realising that the two portions of our code base serve different purposes in our development activities.

So...over to George to fill in some more blanks ;-)

“The Testing Anti-Patterns Drafts” is a collaborative effort between George “spring is overrated” Malamidis and myself which aims to identify cases of Testing gone bad. It consists of a single document that will undergo constant enhancements and modifications, in a “pair-authoring” manner, utilising our respective weblogs as the platform. We hope to get input from anyone following the document, our goal being to produce an interesting resource for the TDD, or Testing Oriented in general community.

Son, it's time to be a geek...

This weekend a shiny new desktop arrived at the Caborn household. After many long months of procrastination, I finally bought a PC for my kids to use. The previous family PC had met an untimely end due to a spilt glass of orange juice about 6 months ago.

While the kids were at school I spent a nice geeky morning setting it up for them and now Heather is happily playing preschooler games and Skyping to my laptop downstairs. Fantastic.

My son wants to be a computer geek like his dad when he grows up. Well, not exactly like me; he plans to drive a DB7 and be a rock star Mondays and Fridays. So, if he's going to lead such a cool life, he needs to learn to program. I'd like to do this with him, in the same way that my dad tried to teach me how to be an engineer by getting out the Meccano set. PCs are easy to get to grips with and Phillip already understands how to put together a logical argument.

At the age of six I think he may be ready to start to learn to program. The only question is: what language should we start with? I posed this question to a number of fellow ThoughtWorkers and, unsurprisingly they came back with a variety of answers!

1. Lisp - "You should teach the guy a proper language and Lisp has everything in it"
2. Logo - "There nothing more rewarding than drawing a picture with a turtle and it made me what I am today!"
3. Flash - "Flash is easy to use and there's instant gratification - it is very visual"
4. Lego Mind Storms - "Its just so cool".
   
5. Java - Java is easy to learn and I'm familiar with it.
6. Javascript - I can't remember why this was a good idea.

So. I'm not certain, but I think I may go with Flash. I like the visual aspects of flash and most of the games and silly things the kids love on the net are made with flash. Wouldn't it be great to write a flash game together including scanned artwork provided by Heather (age 4)....

Time to learn to do stuff with flash. I think I've got about a week before Phillip finishes his latest PS2 game and remembers that dad promised to teach him to code. Should be plenty of time to learn to do something cool in flash!

It's tiddly, it's a wiki...

It's TiddlyWiki!

I'm playing with this at the minute as a replacement for the myriad of little emails, .txt files and other crud that I scatter across my laptop. It seems very nice and extremely user friendly.

The appeal for me at the moment is that I can shove the whole lot onto a USB stick and keep it with me wherever I am.

On graffiti and broken windows

In his book Tipping Point, Malcolm Gladwell describes how graffiti and broken windows can have a dramatic effect on the behaviour of the residents in a city. For those of you who have not read Tipping Point (and I strongly recommend you read the book), the key premises go something like this:

Social change does not occur in the smooth linear way which many people imagine.

Often social values will suddenly transition or ‘tip’ from one state into another. Social change such as crime rates and fashion can often behave in a manner which is similar to the spread of diseases.

A tip can often be achieved by the compound effect of relatively benign factors.

Too much graffiti and too many broken windows can tip a neighbourhood from being a good neighbourhood into a crime ridden no-go area. Additionally, the behaviour of key individuals in a social group can make quite bizarre actions (such as suicide) not just acceptable in a group, but also fashionable and desirable.

In order to repair a problem, you need to perpetually guard against seemingly insignificant factors in order to effect change.

When the new york underground decided to try to clean up the crime and vanadalism on the tube network, they started with the graffiti. The thoery was that by cleaning up the appearance of the trains the feel of the tube network would be improved and people would start to feel more secure and crime would be discouraged. In order to clean up graffiti on their tube network they did it in a staged and defensive manner. A small number of trains were designated as ‘clean trains’. These trains were not allowed to slip and become covered in graffiti even though other trains remained ‘dirty’. The number of clean trains was exteneded in a sustainable manner at the rate the tube system could cope with until the problem tipped and became controllable.

There goes the neighbourhood...

Applied to neighbourhoods, the principal is that an area can hit a point at which there are so many broken windows and walls covered in graffiti that it can change peoples social values. It suddenly becomes 'OK' to break more windows and deface property, and this can move on to more serious crimes being committed once people get a taste for misdemeanours. At this point the neighbourhood has tipped and will rapidly go downhill.

In order to affect change and tip a bad district back into the light, it is necessary to actively repair broken windows and clean up graffiti, because without improving the environment that people live in there will not be enough social impetus to allow the residents to control and discourage antisocial behaviour.

Back in the world of software…

It is interesting to consider if the concepts of the tipping point can be applied to software? I believe that they can. Many applications are perceived by their developers and maintainers to contain either 'good' or 'bad' code. Good code is much cherished by the teams who maintain them – and they bring joy and happiness to the world. Bad code is a millstone around the necks of the maintenance teams and is painful to maintain. But how do we judge good and bad code?

A number of factors can come into play here, but the main dimensions that I feel developers and QA's tend to use to decide if code is good or bad are perceived design quality, and the number of defects in the code base. Poorly designed or ‘smelly’ code is the graffiti of software, and bugs are our broken windows.

Code Quality and graffiti

Code quality is interesting. There are a number of motivating factors which drive for high quality code. Primarily it comes down to the values of the team. If a team values good quality code, then it will attempt to write production code ‘the right way’. However, in any project there are a number of competing drivers which can hamper the realisation of the quality code.

In my mind the biggest anti-quality drivers are:

• The fact that there is a tight deadline.
• Members of the team who do not value quality code.
• Working in a bad district.

The first 2 of these anti quality drivers are common fare. But what about the bad code district? Despite the desire of a team to write good code, they may well struggle if they find themselves in a bad neighbourhood. If the code contains too many code smells (there is a high level of graffiti and broken windows) you may well find that you are producing more smelly code.

Why? Because the developers have lost hope.

The motivation to write good code and behave is much reduced if every preceding developer has treated the code so badly. It becomes very easy for our developer to scrawl on the walls by writing a piece of smelly code or smash a few windows with the odd poorly handled exception because there are so many examples of this about.

The very same developer would be much more likely to run up some scaffolding or lay down the dustsheets with the odd unit test in a good neighbourhood, before going on to build in that new feature.

However, you can get on top of this and agile techniques can be most effective. For a start – agile methodologies value good code. They also build in checks and balances which look to defend against the failure modes of human beings. This is important. If developers were machine like, they would not care if they were working in a bad neighbourhood and so there would be relatively little impact on the quality of new code introduced there. However humans are influenced by their surroundings and your process needs to take this into accounts. A number of XP practices can help here:

• TDD – this is essential to defend against the broken windows initially. You must not add to the problems in your run down area by recklessly adding code. Use the techniques outlined in “Working with Legacy Code” to gradually ‘test infect’ your code base.
• Pair programming – two developers have more courage and are more likely to “do the right thing”. Pair programming is a very effective way of cementing the desired values of the team – you can even use this to ‘inject values’ into the team by clever choices of pairs. If you don’t do pair programming – try using design and code reviews to achieve the same effect.
• Continuous integration – regular builds maximise the return on investment of the TDD by providing lots of regular feedback. This helps to build momentum which is essential to get is you are to tip your code from a run down neighbourhood into an up and coming district.
• Code coverage – not strictly an XP thing – but an important guardian of your defended streets – build this into your CI system.
• Acceptance tests – write them for the new features and get them into the build. If you have time you could try to retro fit them but I have never seen this work well. Better to add them as you add new features or modify bugs.

So – if you have the values and practice in place, how to you ‘tip’ the code base?

If the team values improving the code base and is supported, then you can make tactical improvements. Don’t try to repair the neighbourhood all at once. Rather identify the houses and streets which you repeatedly visit and isolate them. Fix the broken windows by building out unit tests and functional testing to cover just these areas and defend them against the rest of the neighbourhood. Once you have a module under test – defend it with code coverage and automated tests to allow you to spot any breakages and patch them up immediately. Do not allow these clean areas to fall. (todo – relate to the tube trains here).

So – we’re all done then…?

Well, not really. No neighbourhood stays clean and tidy without ongoing effort. You need to actively guard against the problems which can drag your neighbourhood back down to the skids:

• Maverick coders who churn out vast amounts of poor quality code and are not controlled by their managers or team mates.
• Project schedules - prefabs of code are hastily erected to serve a short term need. But are still there 50 years later. This is fine in the short term – but you must plan to repay your design debts in a timely fashion or the surrounding properties in your code will start to suffer.
• False values. You will not succeed if your team does not truly believe in the values they are supposed to believe in. This is the biggest killer of all - as people do not perform at their best when their hearts aren’t in the job at hand.
• Lack of policing. All too often there is not enough effort put into place to police your new district. If you don’t crack down on the perps who break builds and flout the rest with non-TDD coding, then you will suffer from a rising crime rate, more graffiti and more broken windows - "there goes the neighbourhood...”

The inspiration for this entry came from a discussion last Friday in the pub with Jon. He has already blogged about this, but I wasn't going to let that stop me ;-)