On graffiti and broken windows

In his book Tipping Point, Malcolm Gladwell describes how graffiti and broken windows can have a dramatic effect on the behaviour of the residents in a city. For those of you who have not read Tipping Point (and I strongly recommend you read the book), the key premises go something like this:

Social change does not occur in the smooth linear way which many people imagine.

Often social values will suddenly transition or ‘tip’ from one state into another. Social change such as crime rates and fashion can often behave in a manner which is similar to the spread of diseases.

A tip can often be achieved by the compound effect of relatively benign factors.

Too much graffiti and too many broken windows can tip a neighbourhood from being a good neighbourhood into a crime ridden no-go area. Additionally, the behaviour of key individuals in a social group can make quite bizarre actions (such as suicide) not just acceptable in a group, but also fashionable and desirable.

In order to repair a problem, you need to perpetually guard against seemingly insignificant factors in order to effect change.

When the new york underground decided to try to clean up the crime and vanadalism on the tube network, they started with the graffiti. The thoery was that by cleaning up the appearance of the trains the feel of the tube network would be improved and people would start to feel more secure and crime would be discouraged. In order to clean up graffiti on their tube network they did it in a staged and defensive manner. A small number of trains were designated as ‘clean trains’. These trains were not allowed to slip and become covered in graffiti even though other trains remained ‘dirty’. The number of clean trains was exteneded in a sustainable manner at the rate the tube system could cope with until the problem tipped and became controllable.

There goes the neighbourhood...

Applied to neighbourhoods, the principal is that an area can hit a point at which there are so many broken windows and walls covered in graffiti that it can change peoples social values. It suddenly becomes 'OK' to break more windows and deface property, and this can move on to more serious crimes being committed once people get a taste for misdemeanours. At this point the neighbourhood has tipped and will rapidly go downhill.

In order to affect change and tip a bad district back into the light, it is necessary to actively repair broken windows and clean up graffiti, because without improving the environment that people live in there will not be enough social impetus to allow the residents to control and discourage antisocial behaviour.

Back in the world of software…

It is interesting to consider if the concepts of the tipping point can be applied to software? I believe that they can. Many applications are perceived by their developers and maintainers to contain either 'good' or 'bad' code. Good code is much cherished by the teams who maintain them – and they bring joy and happiness to the world. Bad code is a millstone around the necks of the maintenance teams and is painful to maintain. But how do we judge good and bad code?

A number of factors can come into play here, but the main dimensions that I feel developers and QA's tend to use to decide if code is good or bad are perceived design quality, and the number of defects in the code base. Poorly designed or ‘smelly’ code is the graffiti of software, and bugs are our broken windows.

Code Quality and graffiti

Code quality is interesting. There are a number of motivating factors which drive for high quality code. Primarily it comes down to the values of the team. If a team values good quality code, then it will attempt to write production code ‘the right way’. However, in any project there are a number of competing drivers which can hamper the realisation of the quality code.

In my mind the biggest anti-quality drivers are:

• The fact that there is a tight deadline.
• Members of the team who do not value quality code.
• Working in a bad district.

The first 2 of these anti quality drivers are common fare. But what about the bad code district? Despite the desire of a team to write good code, they may well struggle if they find themselves in a bad neighbourhood. If the code contains too many code smells (there is a high level of graffiti and broken windows) you may well find that you are producing more smelly code.

Why? Because the developers have lost hope.

The motivation to write good code and behave is much reduced if every preceding developer has treated the code so badly. It becomes very easy for our developer to scrawl on the walls by writing a piece of smelly code or smash a few windows with the odd poorly handled exception because there are so many examples of this about.

The very same developer would be much more likely to run up some scaffolding or lay down the dustsheets with the odd unit test in a good neighbourhood, before going on to build in that new feature.

However, you can get on top of this and agile techniques can be most effective. For a start – agile methodologies value good code. They also build in checks and balances which look to defend against the failure modes of human beings. This is important. If developers were machine like, they would not care if they were working in a bad neighbourhood and so there would be relatively little impact on the quality of new code introduced there. However humans are influenced by their surroundings and your process needs to take this into accounts. A number of XP practices can help here:

• TDD – this is essential to defend against the broken windows initially. You must not add to the problems in your run down area by recklessly adding code. Use the techniques outlined in “Working with Legacy Code” to gradually ‘test infect’ your code base.
• Pair programming – two developers have more courage and are more likely to “do the right thing”. Pair programming is a very effective way of cementing the desired values of the team – you can even use this to ‘inject values’ into the team by clever choices of pairs. If you don’t do pair programming – try using design and code reviews to achieve the same effect.
• Continuous integration – regular builds maximise the return on investment of the TDD by providing lots of regular feedback. This helps to build momentum which is essential to get is you are to tip your code from a run down neighbourhood into an up and coming district.
• Code coverage – not strictly an XP thing – but an important guardian of your defended streets – build this into your CI system.
• Acceptance tests – write them for the new features and get them into the build. If you have time you could try to retro fit them but I have never seen this work well. Better to add them as you add new features or modify bugs.

So – if you have the values and practice in place, how to you ‘tip’ the code base?

If the team values improving the code base and is supported, then you can make tactical improvements. Don’t try to repair the neighbourhood all at once. Rather identify the houses and streets which you repeatedly visit and isolate them. Fix the broken windows by building out unit tests and functional testing to cover just these areas and defend them against the rest of the neighbourhood. Once you have a module under test – defend it with code coverage and automated tests to allow you to spot any breakages and patch them up immediately. Do not allow these clean areas to fall. (todo – relate to the tube trains here).

So – we’re all done then…?

Well, not really. No neighbourhood stays clean and tidy without ongoing effort. You need to actively guard against the problems which can drag your neighbourhood back down to the skids:

• Maverick coders who churn out vast amounts of poor quality code and are not controlled by their managers or team mates.
• Project schedules - prefabs of code are hastily erected to serve a short term need. But are still there 50 years later. This is fine in the short term – but you must plan to repay your design debts in a timely fashion or the surrounding properties in your code will start to suffer.
• False values. You will not succeed if your team does not truly believe in the values they are supposed to believe in. This is the biggest killer of all - as people do not perform at their best when their hearts aren’t in the job at hand.
• Lack of policing. All too often there is not enough effort put into place to police your new district. If you don’t crack down on the perps who break builds and flout the rest with non-TDD coding, then you will suffer from a rising crime rate, more graffiti and more broken windows - "there goes the neighbourhood...”

The inspiration for this entry came from a discussion last Friday in the pub with Jon. He has already blogged about this, but I wasn't going to let that stop me ;-)