tag:blogger.com,1999:blog-309133422024-03-08T00:05:02.069+00:00No New IdeasThere are no new ideas, just old ones repackaged in new ways. Discuss.Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.comBlogger18125tag:blogger.com,1999:blog-30913342.post-3812455054564918012009-07-19T14:13:00.002+01:002009-07-19T14:18:03.828+01:00What's in a name<span class="Apple-style-span" style="font-family: arial; border-collapse: collapse; font-size: 13px; ">It is really important to call out the names of the big box patterns in a system and the <span class="blsp-spelling-corrected" id="SPELLING_ERROR_0">responsibilities</span> in each box. Even if it is immediately obvious to you how your application should hang together it may not be obvious to all the members of your team. As soon as there is uncertainty people will make guesses and a whole heap of code lands in the wrong place. This is refactoring you didn't need to incur.<br /><br />Once you have identified the aspects of the code that you have names for get a picture on the wall so you can point or stare at it when you are having one of those "where should we put <span class="blsp-spelling-error" id="SPELLING_ERROR_1">xyz</span>?" conversations in your team.<br /><br />Also be vigilant for code escaping it's box - and name the <span class="blsp-spelling-corrected" id="SPELLING_ERROR_2">anti patterns</span> related to each nefarious deed. My favourite is the Fat Controller <span class="blsp-spelling-corrected" id="SPELLING_ERROR_3">anti pattern</span> - where view/service/domain logic wheedles it's way into the controller leading to a spate of Thomas the Tank engine related jokes.<br /><br />Also try to be super consistent with the naming conventions used for your core application structure - it makes a huge difference.</span>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-13098459533749524212009-06-18T15:13:00.006+01:002012-12-19T14:29:20.984+00:00NSynthesis 0.1.1 has escaped!<span class="Apple-style-span" style="font-family: 'Times New Roman';"></span><br />
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">I'm delighted to announce that <a href="http://code.google.com/p/nsynthesis/"><span class="blsp-spelling-error" id="SPELLING_ERROR_0">NSynthesis</span></a> 0.1.1 has finally escaped from my laptop where it has languished for many months. It can be downloaded from <a href="http://nsynthesis.googlecode.com/files/NSynthesis_0.1.1.0.zip">here</a>.</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';"><span class="blsp-spelling-error" id="SPELLING_ERROR_1">NSyntheis</span> is a .NET implementation of the Ruby project, <a href="http://synthesis.rubyforge.org/">Synthesis</a>. The idea is to connect tests which mock expectations to a corresponding test of a real implementation of the mocked call. Why? To give you more confidence that all those <span class="blsp-spelling-error" id="SPELLING_ERROR_2">itty</span> bitty unit tests add up to a comprehensive test suite.</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">Using <span class="blsp-spelling-error" id="SPELLING_ERROR_3">PostSharp</span> to hook into the gap between your test assembly and your production code, <span class="blsp-spelling-error" id="SPELLING_ERROR_4">NSynthesis</span> monitors your unit tests as they run. For each expectation set in your tests, <span class="blsp-spelling-error" id="SPELLING_ERROR_6">NSynthesis</span> demands that you have executed every possible concrete implementation during the same test run. <span class="blsp-spelling-error" id="SPELLING_ERROR_7">NSyntheis</span> uses <span class="blsp-spelling-error" id="SPELLING_ERROR_8">PostSharp</span> to modify the test code only, so there are no changes required to your production code base.</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">Currently, <span class="blsp-spelling-error" id="SPELLING_ERROR_9">NSynthesis</span> requires that you are using the following tools</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<ul>
<li><span class="Apple-style-span" style="font-family: 'Times New Roman';"><span class="blsp-spelling-error" id="SPELLING_ERROR_10">NUnit</span> 2.5</span></li>
<li><span class="Apple-style-span" style="font-family: 'Times New Roman';">Rhino Mocks</span></li>
<li><span class="Apple-style-span" style="font-family: 'Times New Roman';"><span class="blsp-spelling-error" id="SPELLING_ERROR_11">Nant</span></span></li>
</ul>
</div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">The project is definitely at the curiosity stage of development. It needs to be run in on a proper project - so if you fancy giving it a go let me know how you get on with it.</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">The main issue which needs to be fixed next is handling multiple test runs in a build. Currently, there is no memory of the various unit test stages which you may have in a more advanced build (e.g. standalone tests, wired tests, functional tests). This is an issue if the mocked calls and the real code calls are not contained in the same test run.</span></div>
<div style="font: normal normal normal small/normal arial; margin-bottom: 8px; margin-left: 8px; margin-right: 8px; margin-top: 8px;">
<span class="Apple-style-span" style="font-family: 'Times New Roman';">Many thanks to <a href="http://alexscordellis.blogspot.com/">Alex <span class="blsp-spelling-error" id="SPELLING_ERROR_12">Scordellis</span></a>, especially for his help with the evil .NET reflection <span class="blsp-spelling-error" id="SPELLING_ERROR_13">API</span>.</span></div>
</div>
Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com1tag:blogger.com,1999:blog-30913342.post-72242027178003870192008-11-13T21:04:00.007+00:002008-11-13T23:35:30.847+00:00Bowling Cards<p>Recently, my team decided we had to tackle a number of problems with our aging application code base which were causing us pain. There were a number of technology and design problems which needed sorting out. The challenge for us was to continue to deliver new features while taking measures to not only stop the signs of rot, but also to make an effort to improve the code base. If we could improve the code where we were most active we would be able to deliver features more rapidly. </p> <p>The first stage was a technical retrospective, facilitated by Liv, our project manager. The whole team got together and we brainstormed the problems with the code base which were slowing down active development and causing us pain. With everything hurting us out in the open, the team voting for the issues they thought were causing us the most pain. The focus was delivery, we weren't looking to make the code more beautiful for aesthetic reasons; we were trying to reduce friction to help us deliver more code more rapidly. With the key problems agreed, we moved on to discuss how we could fix them. So far so good. </p> <p>Some of our problems could be fixed the traditional agile way write a technical card, estimate the cost and agree who would do the work and when.</p> <p>Others were ongoing "try not to repeat this anti pattern" or "remember to do X" type solutions, so there was no action as such, other than to remember to not do "X" or to do "Y".</p> <p>We also had a third class of problem. Long running repair or rework tasks. For example, the team has switched mocking libraries from NMock to Rhino Mocks. While we find we are more productive when we mock using Rhino mocks, we still have a legacy of old unit tests which periodically bite us when we need to refactor. The idea of scheduling a task to rewrite all our unit tests using Rhino Mocks is just plain wrong for a number of reasons. On humanitarian grounds, I couldn't bring myself to ask a member of my team to spend a week rewriting all our tests - we'd need to put her on suicide watch. Also, from a business value perspective, we only need to rewrite those tests which cover the active areas of the code. The application has been around for a while, and we are only adding features in some areas of the code. If we are not going to be slowed down by NMock riddled tests in a remote area of the code, why waste client money fixing them? Enter the bowling card.</p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ3g66lrL79UW8Cguq1OSAgXlL4pEc6a0p6d7NQ1GvVFq2Khtc3lAqCcPse5f-gqXuogrL0mF2UDR86UM4gsJDfdb-6j6JAQ0jjC3xsgrPC2EdrMfvYELbn9BEOdP3FJFS3PXuTg/s1600-h/in-progress-card%5B4%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="172" alt="in-progress-card" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhH-QqwweShy4y2qqL8ZnzHuyqVv_tXKEu7QD7_EviOKIGQ8I8Oj8JvtTVJ5skGWMG9BSXYdNVcqUX4-MMq7AcuJkHVti3OIHM9u7bOibu1AwHTVhO_dmYFfT_ynbEGudtGTp61_w/?imgmax=800" width="244" align="left" border="0" /></a></p> <p>The anatomy of a bowling card is very simple. The title should be a simple statement of intent. Here, we seek annihilation of all NMock calls in our unit tests. </p> <p>In the middle is a grid, with a box for each step we need to take to win. In this case, we have a box for each test class which includes the NMock library.</p> <p>Anyone gets to work on the bowling card, whenever the time is right. As a team you can decide when this should be. For this card the rules were that if you find yourself maintaining or extending an NMock based class, take the time to fix it by rewriting the tests to use Rhino mocks, or no mock at all. </p> <p>There may or may not be a deadline. For this card, we don't have a deadline. There is no pressing need to replace all the NMock calls, only those in code hotspots. </p> <p>You may or may not schedule the card to be played. For this card we didn't schedule work - there is no real business justification to do that. For another technical card which tracked removing inappropriate methods from a very large legacy repository, we were so slowed down by the problem that we did 'play' the card whenever we had capacity during an iteration. In this case there was clear business value completing the card would not only speed us up, but would allow bigger architectural refactorings to take place later on in the project.</p> <p>The boxes are also very important. Each box is a step towards the goal. We take the steps in two stages.</p> <p><strong>Spares and Strikes</strong></p> <p>When a developer or a pair have fixed one of the steps locally, they get to draw a single line over the box, or a 'spare'. It is not converted to a 'strike' until the code change is checked in, built and tested if appropriate. We have found this to be really useful. The spare is a reminder to check in! With a lot of these tasks, we found it was easy to get carried away and bite off too much. Invariably this led to trouble, broken code and despair, leading to reverting the code and feelings of hopelessness. Once a pair has more than a couple of spares on the board, they start to feel the pressure to 'bank' and tend to check in more frequently.</p> <p>We really like our bowling cards, they are great to remind us to make the world better, and to celebrate progress, no matter how small it feels at times. In fact <a href="http://manicprogrammer.com/cs/blogs/heynemann/archive/2008/11/13/bowling-scorecards-great-agile-practice.aspx">Bernardo</a> and Sinan started another card today to track legacy classes with inadequate unit test coverage - and there are a lot of boxes on that sucker!</p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj112X8QKw4pRwhYujp1uha5au-wqiGOJjM9fNCJAmP0Puy3BrfeSS27uIMAGd347DLASZOLm95EbfLjY44h3WnW5D1wLp2fhf5deNC7a0xytdBN-GIis6cEmnpi8uRoIr_GOFkEA/s1600-h/complete-card%5B3%5D.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="226" alt="complete-card" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpuaAzJrG0dGQLc7gptdAHz8KGV_ksXSJ4fGuLZ5gUZHtO0IjBALGEUCHixPr7cscACJWHWWhRz8R9HCd4IzHxjfFoB_9KFcoMfEMEB4ngi9oCBfr3B3fSMwkQq1SMnHGfFH5q5w/?imgmax=800" width="244" align="left" border="0" /></a>Completing a bowling card is a celebration event. Navya was so happy, she made our first one a party hat! Huzzah!</p> <p>Bowling cards have helped us improve our code. They remind us that <span class="Apple-style-span" style="font-weight: bold;">we chose to fix something,</span> rather than <a href="http://no-new-ideas.blogspot.com/2006/07/on-graffiti-and-broken-windows.html">allow our code to tip</a>. It is going to potentially take a long time, but we can all help by getting strikes when we have the chance. We also have a measure of progress that we can track, and proof that we are making things better one step at a time.</p>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com3tag:blogger.com,1999:blog-30913342.post-65738994215194831042008-11-08T17:02:00.007+00:002008-11-09T11:03:02.695+00:00ActiveRecord lessons learnt: #2 Use lazy loading and batching to improve performance<strike></strike> <p>By default <a href="http://www.castleproject.org/activerecord/index.html">Castle ActiveRecord</a> eagerly loads your object data. For a small set of connected objects, this default behaviour is fine. The problems start when it is possible to navigate from the object that you have loaded to other potentially large object graphs. As an example let's imagine that we are writing some code to track books in our house. Say I want to track which books I have by author, along with the bookshelf that I'm keeping them on. I'd also like to track chapters of interest and which editions of the book that I have. I also want to tag my bookshelves with keywords describing the types of books on the shelf. </p> <p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK-z8rUUHXGPl2lXfZxdhbdeKatLQ4lKiDzXGk0CtgLfRZYdMUntnNXbh6See79pAFYYrxAI8Lhu-4916jU_OyRHiESeUs2jGjoJb1EE1N6-kVTI0sO4ooLU10lTtzbqFcEst-qQ/s1600-h/image%5B15%5D.png"><img style="border-right: 0px; border-top: 0px; border-left: 0px; border-bottom: 0px" height="141" alt="image" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjABUXRVYwfcDoWOqSA2r1E2141of52jU8uC1CKnkcoTEHpKot3pCyVLV5jIrFXb9xnG7P5fuk7yq6m0S_yneT0El27f8HJ1sO2wEjxwpZ8EIfuIBoTS62-rQRYfBVRgzngRW-nPA/?imgmax=800" width="564" border="0" /></a> </p> <p>The Bookshelf and the Author classes both contain collections of Books. Nominally, the Author owns the books and the shelf has a reference to the object. I've also decided that I need to navigate bi-directionally between the book and the author classes. Here's a basic implementation using ActiveRecord.</p> <div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:90d1aa27-974a-4e30-88f8-f9a50ca83bed" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-overflow: auto;color:White;"><div><!-- Code highlighting produced by Actipro CodeHighlighter (freeware) http://www.CodeHighlighter.com/ --><span style="color:#0000FF;">using</span><span style="color:#000000;"> System.Collections.Generic;<br /></span><span style="color:#0000FF;">using</span><span style="color:#000000;"> Castle.ActiveRecord;<br /><br /></span><span style="color:#0000FF;">namespace</span><span style="color:#000000;"> BooksAndStuff.Models<br />{<br /><br />[ActiveRecord(</span><span style="color:#800000;">"</span><span style="color:#800000;">Bookshelf</span><span style="color:#800000;">"</span><span style="color:#000000;">)]<br /></span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Bookshelf<br />{<br /> [PrimaryKey(PrimaryKeyType.HiLo)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> Id { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [HasMany(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(</span><span style="color:#0000FF;">string</span><span style="color:#000000;">), Table </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Tags</span><span style="color:#800000;">"</span><span style="color:#000000;">, ColumnKey </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">BookShelfId</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Cascade </span><span style="color:#000000;">=</span><span style="color:#000000;"> ManyRelationCascadeEnum.AllDeleteOrphan, Element </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">TagName</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyCollections, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.BatchSize)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"> Tags { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [HasMany(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(Book), Table </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Books</span><span style="color:#800000;">"</span><span style="color:#000000;">, ColumnKey </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">BookShelfId</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Cascade </span><span style="color:#000000;">=</span><span style="color:#000000;"> ManyRelationCascadeEnum.None,<br /> Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyCollections, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.BatchSize)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;"> Books { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br />}<br /><br />[ActiveRecord(</span><span style="color:#800000;">"</span><span style="color:#800000;">Books</span><span style="color:#800000;">"</span><span style="color:#000000;">)]<br /></span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Book<br />{<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> Book() {}<br /><br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> Book(</span><span style="color:#0000FF;">string</span><span style="color:#000000;"> title, Author author, </span><span style="color:#0000FF;">string</span><span style="color:#000000;"> isbn,<br /> IList</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"> chapters, IList</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"> editions) {<br /> Title </span><span style="color:#000000;">=</span><span style="color:#000000;"> title;<br /> Author </span><span style="color:#000000;">=</span><span style="color:#000000;"> author;<br /> Chapters </span><span style="color:#000000;">=</span><span style="color:#000000;"> chapters;<br /> ISBN </span><span style="color:#000000;">=</span><span style="color:#000000;"> isbn;<br /> Editions </span><span style="color:#000000;">=</span><span style="color:#000000;"> editions;<br /> }<br /><br /> [PrimaryKey(PrimaryKeyType.HiLo)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> Id { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [Property(NotNull </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">true</span><span style="color:#000000;">)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">string</span><span style="color:#000000;"> Title { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [HasMany(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(</span><span style="color:#0000FF;">string</span><span style="color:#000000;">), Table </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Editions</span><span style="color:#800000;">"</span><span style="color:#000000;">, ColumnKey </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">BookId</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Access </span><span style="color:#000000;">=</span><span style="color:#000000;"> PropertyAccess.Property, Element </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Edition</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyCollections, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.BatchSize)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"> Editions { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /><br /> [HasMany(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(</span><span style="color:#0000FF;">string</span><span style="color:#000000;">), Table </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Chapters</span><span style="color:#800000;">"</span><span style="color:#000000;">, ColumnKey </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">BookId</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Access </span><span style="color:#000000;">=</span><span style="color:#000000;"> PropertyAccess.Property, Element </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Chapter</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyCollections, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.BatchSize)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"> Chapters { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [Property(NotNull </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">true</span><span style="color:#000000;">)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">string</span><span style="color:#000000;"> ISBN { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /><br /> [BelongsTo(NotNull </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">true</span><span style="color:#000000;">)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> Author Author { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br />}<br /><br /><br />[ActiveRecord(</span><span style="color:#800000;">"</span><span style="color:#800000;">Authors</span><span style="color:#800000;">"</span><span style="color:#000000;">, Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyAuthor, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.AuthorBatchSize)]<br /></span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Author<br />{<br /> [PrimaryKey(PrimaryKeyType.HiLo)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">virtual</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> Id { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [Property(NotNull </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">true</span><span style="color:#000000;">)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">virtual</span><span style="color:#000000;"> </span><span style="color:#0000FF;">string</span><span style="color:#000000;"> Name { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br /> [HasMany(Cascade </span><span style="color:#000000;">=</span><span style="color:#000000;"> ManyRelationCascadeEnum.AllDeleteOrphan,<br /> Lazy </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.LazyCollections, BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> Switches.BatchSize)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">virtual</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;"> Books { </span><span style="color:#0000FF;">get</span><span style="color:#000000;">; </span><span style="color:#0000FF;">set</span><span style="color:#000000;">; }<br /><br />}<br /><br /></span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">static</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Switches<br />{<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyCollections </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">false</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyAuthor </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">false</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">1</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> AuthorBatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">1</span><span style="color:#000000;">;<br />}<br />}<br /></span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>I wouldn't normally have a switches class, but it is very useful for the next part. We are going to change the lazy loading and batch size properties of the active record attributes to understand how it influences the way NHibernate fetches the data from the database. To get a feel for the behaviour of NHibernate, we will need some test data. Say that we have a single shelf with 3 books on it, each with a different author. We have another 2 books by one of our authors, but they are not on a shelf (I lent them to Steve). </p><p><img src="http://www.gliffy.com/pubdoc/1534245/M.jpg" /><br /></p><p>To test this we need to start up active record:<br /></p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:42f2a548-6098-4a43-8f9c-0de497952e98" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; width: 952px; padding-top: 0px"><pre style="background-overflow: auto;color:White;"><div><span style="color:#000000;"> [TestFixtureSetUp]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">void</span><span style="color:#000000;"> FixtureSetUp()<br /> {<br /> var connectionString </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">@"</span><span style="color:#800000;">Data Source=localhost;initial catalog=BooksAndStuff;<br /> user=BooksAndStuff_user;password=Password01</span><span style="color:#800000;">"</span><span style="color:#000000;">;<br /> var properties </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Dictionary</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">, </span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;"><br /> {<br /> {</span><span style="color:#800000;">"</span><span style="color:#800000;">show_sql</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">true</span><span style="color:#800000;">"</span><span style="color:#000000;">},<br /> {</span><span style="color:#800000;">"</span><span style="color:#800000;">connection.driver_class</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">NHibernate.Driver.SqlClientDriver</span><span style="color:#800000;">"</span><span style="color:#000000;">},<br /> {</span><span style="color:#800000;">"</span><span style="color:#800000;">dialect</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">NHibernate.Dialect.MsSql2005Dialect</span><span style="color:#800000;">"</span><span style="color:#000000;">},<br /> {</span><span style="color:#800000;">"</span><span style="color:#800000;">connection.provider</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">NHibernate.Connection.DriverConnectionProvider</span><span style="color:#800000;">"</span><span style="color:#000000;">},<br /> {</span><span style="color:#800000;">"</span><span style="color:#800000;">connection.connection_string</span><span style="color:#800000;">"</span><span style="color:#000000;">, connectionString}<br /> };<br /><br /> var source </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> InPlaceConfigurationSource();<br /> source.Add(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(ActiveRecordBase), properties);<br /> ActiveRecordStarter.Initialize(source, </span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(BookShelf).Module.GetTypes());<br /> ActiveRecordStarter.CreateSchema();<br /> }<br /><br /> [SetUp]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">void</span><span style="color:#000000;"> SetUp()<br /> {<br /> ActiveRecordStarter.CreateSchema();<br /> }</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>This method creates this in the database:</p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:94669d1c-5696-4cb1-a8d4-7d82c3ea9a7f" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-overflow: auto;color:White;"><div><span style="color:#000000;"> </span><span style="color:#0000FF;">private</span><span style="color:#000000;"> </span><span style="color:#0000FF;">void</span><span style="color:#000000;"> GenerateTestData() {<br /> var lewisCarol </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Author {Name </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Lewis Carol</span><span style="color:#800000;">"</span><span style="color:#000000;">};<br /> var xmasCarol </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Author {Name </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Xmas Carol</span><span style="color:#800000;">"</span><span style="color:#000000;">};<br /> var luisLuisCarol </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Author {Name </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800000;">"</span><span style="color:#800000;">Luis Luis Carol</span><span style="color:#800000;">"</span><span style="color:#000000;">};<br /><br /> var chapters </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;">{ </span><span style="color:#800000;">"</span><span style="color:#800000;">Chapter 1</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">Chapter 2</span><span style="color:#800000;">"</span><span style="color:#000000;"> };<br /> var editions </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;">{</span><span style="color:#800000;">"</span><span style="color:#800000;">hard back</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">spanish</span><span style="color:#800000;">"</span><span style="color:#000000;">};<br /><br /> var book1 </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Book(</span><span style="color:#800000;">"</span><span style="color:#800000;">Alice in Wonderland</span><span style="color:#800000;">"</span><span style="color:#000000;">, lewisCarol,</span><span style="color:#800000;">"</span><span style="color:#800000;">xxx-yyy</span><span style="color:#800000;">"</span><span style="color:#000000;">, chapters, editions);<br /> var book2 </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Book(</span><span style="color:#800000;">"</span><span style="color:#800000;">Alice in Wonderland II - Revenge of the Cheshire Cat</span><span style="color:#800000;">"</span><span style="color:#000000;">, xmasCarol,</span><span style="color:#800000;">"</span><span style="color:#800000;">xxx-yyy</span><span style="color:#800000;">"</span><span style="color:#000000;">, chapters, editions);<br /> var book3 </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Book(</span><span style="color:#800000;">"</span><span style="color:#800000;">Not well known</span><span style="color:#800000;">"</span><span style="color:#000000;">,lewisCarol, </span><span style="color:#800000;">"</span><span style="color:#800000;">xxx-yyy</span><span style="color:#800000;">"</span><span style="color:#000000;">, chapters, editions);<br /> var book4 </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Book(</span><span style="color:#800000;">"</span><span style="color:#800000;">Not well known either</span><span style="color:#800000;">"</span><span style="color:#000000;">, lewisCarol, </span><span style="color:#800000;">"</span><span style="color:#800000;">xxx-yyy</span><span style="color:#800000;">"</span><span style="color:#000000;">, chapters, editions);<br /> var book5 </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Book(</span><span style="color:#800000;">"</span><span style="color:#800000;">Some book</span><span style="color:#800000;">"</span><span style="color:#000000;">,luisLuisCarol,</span><span style="color:#800000;">"</span><span style="color:#800000;">xxx-yyy</span><span style="color:#800000;">"</span><span style="color:#000000;">, chapters, editions);<br /><br /> lewisCarol.Books </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;"> {book1, book3, book4};<br /> xmasCarol.Books </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;">{book2};<br /> luisLuisCarol.Books </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;">{ book5 };<br /><br /> var bookshelf </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> Bookshelf<br /> {<br /> Tags </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#0000FF;">string</span><span style="color:#000000;">></span><span style="color:#000000;">{</span><span style="color:#800000;">"</span><span style="color:#800000;">Fantasy</span><span style="color:#800000;">"</span><span style="color:#000000;">, </span><span style="color:#800000;">"</span><span style="color:#800000;">Old</span><span style="color:#800000;">"</span><span style="color:#000000;">},<br /> Books </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">new</span><span style="color:#000000;"> List</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;"> {book1, book2, book5}<br /> };<br /><br /> </span><span style="color:#0000FF;">using</span><span style="color:#000000;"> (</span><span style="color:#0000FF;">new</span><span style="color:#000000;"> SessionScope())<br /> {<br /> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">Author</span><span style="color:#000000;">></span><span style="color:#000000;">.Save(lewisCarol);<br /> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">Author</span><span style="color:#000000;">></span><span style="color:#000000;">.Save(xmasCarol);<br /> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">Author</span><span style="color:#000000;">></span><span style="color:#000000;">.Save(luisLuisCarol);<br /> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">BookShelf</span><span style="color:#000000;">></span><span style="color:#000000;">.Save(bookshelf);<br /> }<br /> }<br /></span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>Now we have everything in place, we can write a test to load in the data:<br /></p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:22a4dab2-2d1f-4716-809e-f3291ea6a56c" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-overflow: auto;color:White;"><div><span style="color:#000000;"> [Test]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">void</span><span style="color:#000000;"> PlayingWithBatchFetchingAndLazyLoading()<br /> {<br /> GenerateTestData();<br /><br /> </span><span style="color:#0000FF;">using</span><span style="color:#000000;"> (</span><span style="color:#0000FF;">new</span><span style="color:#000000;"> SessionScope())<br /> {<br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">*******************</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">Batch Size:{0} LazyCollections:{1} LazyAuthor:{2}</span><span style="color:#800000;">"</span><span style="color:#000000;">,<br /> Switches.BatchSize, Switches.LazyCollections, Switches.LazyAuthor);<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nLoading my book shelves</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> var shelves </span><span style="color:#000000;">=</span><span style="color:#000000;"> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">Bookshelf</span><span style="color:#000000;">></span><span style="color:#000000;">.FindAll();<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nCounting books on shelf: </span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">shelves[0].Books.Count =</span><span style="color:#800000;">"</span><span style="color:#000000;"> </span><span style="color:#000000;">+</span><span style="color:#000000;"> shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books.Count);<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nChecking I have the right book</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">\tshelves[0].Books[0].Id:</span><span style="color:#800000;">"</span><span style="color:#000000;"> </span><span style="color:#000000;">+</span><span style="color:#000000;"> shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Id);<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nCounting the book's chapters</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Chapters.Count, Is.EqualTo(</span><span style="color:#800080;">2</span><span style="color:#000000;">));<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nHow many books does the first book's author have to his name?</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Author.Books.Count, Is.EqualTo(</span><span style="color:#800080;">3</span><span style="color:#000000;">));<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nHow many books does the third book's author have to his name?</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">2</span><span style="color:#000000;">].Author.Books.Count, Is.EqualTo(</span><span style="color:#800080;">1</span><span style="color:#000000;">));<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nHow many books does the second book's author have to his name?</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">1</span><span style="color:#000000;">].Author.Books.Count, Is.EqualTo(</span><span style="color:#800080;">1</span><span style="color:#000000;">));<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nWhat is the count of the chapters in a given book?</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Author.Books[</span><span style="color:#800080;">1</span><span style="color:#000000;">].Chapters.Count, Is.EqualTo(</span><span style="color:#800080;">2</span><span style="color:#000000;">));<br /><br /> Console.WriteLine(</span><span style="color:#800000;">"</span><span style="color:#800000;">***\nHow many chapters does the third book have?</span><span style="color:#800000;">"</span><span style="color:#000000;">);<br /> Assert.That(shelves[</span><span style="color:#800080;">0</span><span style="color:#000000;">].Books[</span><span style="color:#800080;">2</span><span style="color:#000000;">].Chapters.Count, Is.EqualTo(</span><span style="color:#800080;">2</span><span style="color:#000000;">));<br /> }<br /> }</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>When you run this, you should see the following output from NHibernate:</p><p> <span style="font-family:Courier;">*******************<br />Batch Size:1 LazyCollections:False LazyAuthor:False<br />***<br />Loading my book shelves<br />NHibernate: SELECT this_.Id as Id0_0_ FROM BookShelf this_<br />NHibernate: SELECT books0_.BookShelfId as BookShel5___2_, books0_.Id as Id2_, books0_.Id as Id2_1_, books0_.Title as Title2_1_, books0_.ISBN as ISBN2_1_, books0_.Author as Author2_1_, author1_.Id as Id5_0_, author1_.Name as Name5_0_ FROM Books books0_ inner join Authors author1_ on books0_.Author=author1_.Id WHERE books0_.BookShelfId=@p0; @p0 = '98305'<br />NHibernate: SELECT chapters0_.BookId as BookId__0_, chapters0_.Chapter as Chapter0_ FROM Chapters chapters0_ WHERE chapters0_.BookId=@p0; @p0 = '65541'<br />NHibernate: SELECT editions0_.BookId as BookId__0_, editions0_.Edition as Edition0_ FROM Editions editions0_ WHERE editions0_.BookId=@p0; @p0 = '65541'<br />NHibernate: SELECT books0_.Author as Author__1_, books0_.Id as Id1_, books0_.Id as Id2_0_, books0_.Title as Title2_0_, books0_.ISBN as ISBN2_0_, books0_.Author as Author2_0_ FROM Books books0_ WHERE books0_.Author=@p0; @p0 = '32771'<br />NHibernate: SELECT chapters0_.BookId as BookId__0_, chapters0_.Chapter as Chapter0_ FROM Chapters chapters0_ WHERE chapters0_.BookId=@p0; @p0 = '65540'<br />NHibernate: SELECT editions0_.BookId as BookId__0_, editions0_.Edition as Edition0_ FROM Editions editions0_ WHERE editions0_.BookId=@p0; @p0 = '65540'<br />NHibernate: SELECT books0_.Author as Author__1_, books0_.Id as Id1_, books0_.Id as Id2_0_, books0_.Title as Title2_0_, books0_.ISBN as ISBN2_0_, books0_.Author as Author2_0_ FROM Books books0_ WHERE books0_.Author=@p0; @p0 = '32770'<br />NHibernate: SELECT chapters0_.BookId as BookId__0_, chapters0_.Chapter as Chapter0_ FROM Chapters chapters0_ WHERE chapters0_.BookId=@p0; @p0 = '65537'<br />NHibernate: SELECT editions0_.BookId as BookId__0_, editions0_.Edition as Edition0_ FROM Editions editions0_ WHERE editions0_.BookId=@p0; @p0 = '65537'<br />NHibernate: SELECT books0_.Author as Author__1_, books0_.Id as Id1_, books0_.Id as Id2_0_, books0_.Title as Title2_0_, books0_.ISBN as ISBN2_0_, books0_.Author as Author2_0_ FROM Books books0_ WHERE books0_.Author=@p0; @p0 = '32769'<br />NHibernate: SELECT chapters0_.BookId as BookId__0_, chapters0_.Chapter as Chapter0_ FROM Chapters chapters0_ WHERE chapters0_.BookId=@p0; @p0 = '65539'<br />NHibernate: SELECT editions0_.BookId as BookId__0_, editions0_.Edition as Edition0_ FROM Editions editions0_ WHERE editions0_.BookId=@p0; @p0 = '65539'<br />NHibernate: SELECT chapters0_.BookId as BookId__0_, chapters0_.Chapter as Chapter0_ FROM Chapters chapters0_ WHERE chapters0_.BookId=@p0; @p0 = '65538'<br />NHibernate: SELECT editions0_.BookId as BookId__0_, editions0_.Edition as Edition0_ FROM Editions editions0_ WHERE editions0_.BookId=@p0; @p0 = '65538'<br />NHibernate: SELECT tags0_.BookShelfId as BookShel1___0_, tags0_.TagName as TagName0_ FROM Tags tags0_ WHERE tags0_.BookShelfId=@p0; @p0 = '98305'<br />***<br />Counting books on shelf: <br />shelves[0].Books.Count =3<br />***<br />Checking I have the right book<br /> shelves[0].Books[0].Id:65537<br />***<br />Counting the book's chapters<br />***<br />How many books does the first book's author have to his name?<br />***<br />How many books does the third book's author have to his name?<br />***<br />How many books does the second book's author have to his name?<br />***<br />What is the count of the chapters in a given book?<br />***<br />How many chapters does the third book have?</span></p><p>Blimey - that was a lot of SQL! 16 Select statements. Also, notice how it was <strong>all</strong> executed on the initial load of the Bookshelf. The logic for loading ran something like this:</p><ol> <li>Load all the data from the Bookshelf table for all the bookshelves</li> <li>For each bookshelf <ol> <li>Load in all the data from the books and authors table for each book on the shelf and its corresponding author</li> <li>For each book on the shelf <ol> <li>Load all the Chapters for the first book on the shelf</li> <li>Load all the editions for the first book in the shelf</li></ol></li> <li>For each author <ol> <li>Load all the data from the books table for the author</li> <li>Repeat the book loading logic for every book belonging to the author which was not on the shelf</li></ol></li> <li>Load in all the tags for the shelf</li> </ol><br /></li></ol><p>All that so that we can ask for the first shelf. We haven't even begun to touch the object yet. Why did NHibernate do this? All of these objects can theoretically be reached from the bookshelf. Because we may want to touch any of these objects after we have retrieved the BookShelf from the database, NHibernate decided it better go and grab the lot so that we could get to these objects if we had to. This is eager loading, and is the default behaviour of ActiveRecord unless you explicitly instruct it otherwise. Notice also, that we have an N+1 selects problem too. There is a single select statement being issued for every collection for every object. This is a a very expensive way to fetch data from the database.</p><p>The key to optimising the fetch behaviour is how you use the the Lazy and BatchSize properties of the various ActiveRecord attributes.</p><p><strong>Removing N+1 Select with BatchSize</strong></p><p>The BatchSize property can be used on collections or on ActiveRecord classed directly. When you use the setting on a collection, NHibernate will attempt to populate the collection for [BatchSize] of the objects which NHibernate is aware of an has not populated yet. </p><p>We'll start by adding batching</p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:4707de3f-2137-42a0-ac0f-c71eb557a62e" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-overflow: auto;color:White;"><div><span style="color:#000000;"> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">static</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Switches<br />{<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyCollections </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">false</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyAuthor </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">false</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">2</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> AuthorBatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">10</span><span style="color:#000000;">;<br />}</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p><span style="font-family:Courier;">Batch Size:2 LazyCollections:False LazyAuthor:False<br />***<br />Loading my book shelves <br />SELECT this_.Id as Id3_0_ FROM BookShelf this_<br />SELECT books0_.BookShelfId as BookShel5___2_, books0_.Id as ...<br />SELECT chapters0_.BookId...WHERE chapters0_.BookId in (@p0, @p1); @p0 = '65541', @p1 = '65540'<br />SELECT editions0_.BookId...WHERE editions0_.BookId in (@p0, @p1); @p0 = '65541', @p1 = '65540'<br />SELECT books0_.Author...WHERE books0_.Author in (@p0, @p1); @p0 = '32771', @p1 = '32770'<br />SELECT chapters0_.BookId...WHERE chapters0_.BookId=@p0; @p0 = '65537'<br />SELECT editions0_.BookId...WHERE editions0_.BookId=@p0; @p0 = '65537'<br />SELECT books0_.Author...WHERE books0_.Author=@p0; @p0 = '32769'<br />SELECT chapters0_.BookId...WHERE chapters0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />SELECT editions0_.BookId...WHERE editions0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />SELECT tags0_.BookShelfId...WHERE tags0_.BookShelfId=@p0; @p0 = '98305'<br />***</span></p><p>That improved the behaviour somewhat. We're down to 11 statements. NHibernate is now resolving the objects in the collections 2 at a time. This is not very useful as we typically have more than 2 books on a shelf, so we have 2 trips round the loop to load in all the data for our shelf. If we up the BatchSize to 5 we can get everything in 7 statements:</p><p><span style="font-family:Courier;">Batch Size:5 LazyCollections:False LazyAuthor:False<br />***<br />Loading my book shelves<br />SELECT this_.Id as Id3_0_ FROM BookShelf this_<br />SELECT books0_.BookShelfId...WHERE books0_.BookShelfId=@p0; @p0 = '98305'<br />SELECT chapters0_.BookId..WHERE chapters0_.BookId in (@p0, @p1, @p2); @p0 = '65541', @p1 = '65537', @p2 = '65540'<br />SELECT editions0_.BookId...WHERE editions0_.BookId in (@p0, @p1, @p2); @p0 = '65541', @p1 = '65537', @p2 = '65540'<br />SELECT books0_.Author...WHERE books0_.Author in (@p0, @p1, @p2); @p0 = '32771', @p1 = '32769', @p2 = '32770'<br />SELECT chapters0_.BookId...WHERE chapters0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />SELECT editions0_.BookId...WHERE editions0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />SELECT tags0_.BookShelfId...WHERE tags0_.BookShelfId=@p0; @p0 = '98305'<br />***</span></p><p>Bear in mind that the BatchSize only influences how many of the objects which will be populated in the current known set of objects. In the example above, even though we only have 5 books we still have make two trips to the database to load all the books in. The first time NHibernate decides it needs to load in a Book is when it loads the shelf. At this point in time, there are only 3 <strong>known books</strong> so they are fetched. Later on, while loading in the authors, the 2 remaining books are identified and fetched in a second batch.</p><p><span class="Apple-style-span" style="font-weight: bold; ">Using Lazy Loading</span><br /></p><p>The biggest problem with the fetch patterns above is the large amount of data being retrieved which may not be required. This is potentially very wasteful. Lazy loading allows you to control this. With our current object model, it probably doesn't make sense to always pull in the author (and the related books of the author) every time we load a bookshelf. How about we look at the results of lazy loading the author.</p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:f088fe6c-6eb3-4d3d-ae78-c733b52f1ab9" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre color="White" style="background-overflow: auto;"><div><span style="color:#000000;"> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">static</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> Switches<br />{<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyCollections </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">false</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">bool</span><span style="color:#000000;"> LazyAuthor </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#0000FF;">true</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> BatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">5</span><span style="color:#000000;">;<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">const</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> AuthorBatchSize </span><span style="color:#000000;">=</span><span style="color:#000000;"> </span><span style="color:#800080;">10</span><span style="color:#000000;">;<br />}</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p><span style="font-family:Courier;">*******************<br />Batch Size:5 LazyCollections:False LazyAuthor:True<br />***<br />Loading my book shelves<br />SELECT this_.Id as Id3_0_ FROM BookShelf this_<br />SELECT books0_.BookShelfId... WHERE books0_.BookShelfId=@p0; @p0 = '98305'<br />SELECT chapters0_.BookId ... WHERE chapters0_.BookId in (@p0, @p1, @p2); @p0 = '65541', @p1 = '65537', @p2 = '65540'<br />SELECT editions0_.BookId ... WHERE editions0_.BookId in (@p0, @p1, @p2); @p0 = '65541', @p1 = '65537', @p2 = '65540'<br />SELECT tags0_.BookShelfId ... WHERE tags0_.BookShelfId=@p0; @p0 = '98305'<br />***<br />Counting books on shelf: <br />shelves[0].Books.Count =3<br />***<br />Checking I have the right book<br /> shelves[0].Books[0].Id:65537<br />***<br />Counting the book's chapters<br />***<br />How many books does the first book's author have to his name?<br />SELECT author0_.Id ... WHERE author0_.Id in (@p0, @p1, @p2); @p0 = '32769', @p1 = '32770', @p2 = '32771'<br />SELECT books0_.Author ... WHERE books0_.Author in (@p0, @p1, @p2); @p0 = '32771', @p1 = '32769', @p2 = '32770'<br />SELECT chapters0_.BookId ... WHERE chapters0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />SELECT editions0_.BookId ... WHERE editions0_.BookId in (@p0, @p1); @p0 = '65539', @p1 = '65538'<br />***<br />How many books does the third book's author have to his name?<br />***<br />How many books does the second book's author have to his name?<br />***<br />What is the count of the chapters in a given book?<br />***<br />How many chapters does the third book have?</span></p><p>This has dramatically changed the way we retrieve data from the database. Now we do not load any of the books which are not on the shelf until we touch the author property of the book. The initial load is much cheaper, although we still pull in a load of data which is not required. How about we lazy load the collections as well?</p><p><span style="font-family:Courier;">*******************<br />Batch Size:5 LazyCollections:True LazyAuthor:True<br />***<br />Loading my book shelves<br />NHibernate: SELECT this_.Id as Id3_0_ FROM BookShelf this_<br />***<br />Counting books on shelf:<br />SELECT books0_.BookShelfId... WHERE books0_.BookShelfId=@p0; @p0 = '98305'<br />shelves[0].Books.Count =3<br />***<br />Checking I have the right book<br />shelves[0].Books[0].Id:65537<br />***<br />Counting the book's chapters<br />SELECT chapters0_.BookId... WHERE chapters0_.BookId in (@p0, @p1, @p2); @p0 = '65537', @p1 = '65540', @p2 = '65541'<br />***<br />How many books does the first book's author have to his name?<br />SELECT ... WHERE author0_.Id in (@p0, @p1, @p2); @p0 = '32769', @p1 = '32770', @p2 = '32771'<br />SELECT books0_.Author... WHERE books0_.Author in (@p0, @p1, @p2); @p0 = '32769', @p1 = '32770', @p2 = '32771'<br />***<br />How many books does the third book's author have to his name?<br />***<br />How many books does the second book's author have to his name?<br />***<br />What is the count of the chapters in a given book?<br />NHibernate: SELECT chapters0_.BookId... WHERE chapters0_.BookId in (@p0, @p1); @p0 = '65538', @p1 = '65539'<br />***<br />How many chapters does the third book have?</span> </p><p>As you can see, the up front cost of loading the bookshelves is now very cheap, we are progressively loading in data as we touch more and more of the object graph. I would suggest that this is generally the best behaviour. In this case we never retrieve the tags or editions from the database because we don't touch them. If another method wants to load a bookshelf to edit its tags, we won't incur the cost of loading all the books on the shelf (and there could be 100's of those).</p><p>In summary, I'd recommend that you default to setting Lazy loading and batch fetching for all collections and consider lazy loading large aggregate classes - such as the Author in this example, so that you don't inadvertently pull out huge chunks of the database. Also, keep watching your SQL output by setting show_sql to true now and again. Quite innocent seeming changes to your domain model can quickly magnify the number of objects that can be navigated to from another domain object. If you don't have a Lazy constraint between your objects to act as an NHibernate firebreak, then you will cause the amount of SQL issued to increase dramatically.</p><p>Don't forget that lazy loading can only work when you remain inside the hibernate session. If you detach your object from the session then you will get a LazyInitializationException thrown when you attempt to access a collection of object that has not been initialised.</p>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-39946777144658819572008-11-05T13:51:00.007+00:002008-11-05T14:46:13.685+00:00ActiveRecord lessons learnt: #1 Never forget there's a database<p>We have been using Castle's Active Record very heavily on my current project we have recently spent a lot of time performance tuning the performance of the application and its use of ActiveRecord. We learnt a lot over the last few of weeks and in the next few posts I'll try to capture some of these lessons.</p> <p>ActiveRecord is a great abstraction of the database for a .NET application. NHibernate and Hibernate are OK, but leave you with a legacy of XML mapping files in your application, which is no fun at all. The appeal of ActiveRecord over vanilla NHibernate was the chance to shed all that XML cruft. Now, with a minimal few simple attributes and a simple repository class I can transform my plain old C# object into a persistent domain object with ease. It is almost as if the database was not there at all...</p> <div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:1a99e986-fbac-47b3-a435-443a5e6431b9" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;;overflow: auto;"><div><!-- Code highlighting produced by Actipro CodeHighlighter (freeware) http://www.CodeHighlighter.com/ --><span style="color:#000000;">[ActiveRecord]<br /></span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> MyAggregate<br />{<br /> </span><span style="color:#0000FF;">private</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> id;<br /> </span><span style="color:#0000FF;">private</span><span style="color:#000000;"> </span><span style="color:#0000FF;">readonly</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#000000;">SomeObject</span><span style="color:#000000;">></span><span style="color:#000000;"> items;<br /><br /> [PrimaryKey(Access </span><span style="color:#000000;">=</span><span style="color:#000000;"> PropertyAccess.FieldCamelcase, UnsavedValue </span><span style="color:#000000;">=</span><span style="color:#000000;"> Entity.IdUnassignedString)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">int</span><span style="color:#000000;"> Id{</span><span style="color:#0000FF;">get</span><span style="color:#000000;"> { </span><span style="color:#0000FF;">return</span><span style="color:#000000;"> id; } }<br /><br /> [HasMany(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(SomeObject), Access</span><span style="color:#000000;">=</span><span style="color:#000000;">PropertyAccess.FieldCamelCase)]<br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#000000;">SomeObject</span><span style="color:#000000;">></span><span style="color:#000000;"> SomeItems{ </span><span style="color:#0000FF;">get</span><span style="color:#000000;"> { </span><span style="color:#0000FF;">return</span><span style="color:#000000;"> items; } }<br />}</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>Look at that! Just three Attributes and my object is ready for the database. Marvelous!</p><p>This has been the source of many problems for us. It was just too easy to work with the database. In the green field days of the project, you could almost forget that the database existed at all. We had ActiveRecord auto generate the database, and we wrote test fixtures which could create test data in memory or in the database. This was superb, as we could write standalone unit tests to drive out the behaviour of the domain and application components, soon we had hundreds of fast database free unit tests. We could also write wired tests for our repositories to prove that we could persist domain objects to and read them back from SQL server. The same test fixtures could be reused in the automated acceptance tests to modify the application data as required.</p><p>Fantastic! So what went wrong? Well, we built a slow app, at least it started out slow when it hit UAT (it's much, much faster now). We fell victim to Joel's <a href="http://www.joelonsoftware.com/articles/LeakyAbstractions.html">Law of Leaky Abstractions</a> which states that "All non-trivial abstractions, to some degree, are leaky". In this case, we have ActiveRecord abstracting NHibernate which in turn abstracts the database access. The minute you forget this, you will get bitten. The Hibernate team certainly acknowledge this. Just three pages into the first chapter of <a href="http://www.manning.com/bauer2/">Java Persistence with Hibernate</a> Christian Bauer and Gavin King remind you that:</p><blockquote> <p>"To use Hibernate effectively, a solid understanding of the relational model and SQL is a prerequisite. You need to understand the relational model and topics such as normalization to guarantee the the integrity of your data, and you'll need to use your knowledge of SQL to tune the performance of your Hibernate application" </p></blockquote><p>In other words, try to remember that you are still talking to a relational database; we're just making it easier for you to do so from your OO world.</p><p>I really like ActiveRecord and I would recommend using it on .NET projects when there is the need an OR Mapping tool. The fault lay was us, the developers. We didn't pay enough attention to the fact that there was a database at the end of the calls that the repository was making. We didn't spot that some of the calls which our test code was making were potentially very expensive in real usage scenarios. And so we got bitten by the law of leaky abstractions, which manifest itself through a number of bad usage patterns which caused immediate performance problems.</p><p><strong>Avoidable Mistake - Absence of good quality test data</strong></p><p>We took too long time to recognise we had performance problems because we didn't work hard enough to obtain truly representative test data for our app. If we had made more of an effort to use 'real' data sooner, then we would have spotted much more, much earlier in our project's lifecycle. There is no substitute for realistic data. Get or generate some and plan to integrate it into your application build process. Make it easy for developers to use realistic data. We now have a nant target which restores a UAT or production database onto our developer machines and upgrades it to the latest version using <a href="http://dbdeploy.com/">dbdeploy.NET</a>. This is great as it means we can also automate testing our data and schema migration scripts with a production-like database.</p><p><strong><strong>Avoidable </strong>Mistake - Adoption of the "select all and filter with a bit of LINQ" anti pattern</strong></p><p>This is a real killer. The code snippet below illustrates the problem</p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:572a7a91-7a30-47e5-b754-8fab43dfcde1" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;;overflow: auto;"><div><!-- Code highlighting produced by Actipro CodeHighlighter (freeware) http://www.CodeHighlighter.com/ --><span style="color:#0000FF;">public</span><span style="color:#000000;"> Book FindBooksWitTitle(</span><span style="color:#0000FF;">string</span><span style="color:#000000;"> title)<br />{<br /> </span><span style="color:#0000FF;">return</span><span style="color:#000000;"> repository.AllBooks().Select(book</span><span style="color:#000000;">=></span><span style="color:#000000;">book.Title </span><span style="color:#000000;">==</span><span style="color:#000000;"> title).FirstOrDefault();<br />}</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>This is fine from a C# perspective, and is nice, clean, expressive code. The problem is that we had to pull all the books out of the database to run the query. Fine with a small test database, but very bad with the Amazon book catalogue.</p><p>Provide a richer API through the repository instead and push the filtering to the database:</p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:3f6c2170-dfba-46da-9615-b9950fcba6bd" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;;overflow: auto;"><div><!-- Code highlighting produced by Actipro CodeHighlighter (freeware) http://www.CodeHighlighter.com/ --><span style="color:#000000;"> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> </span><span style="color:#0000FF;">class</span><span style="color:#000000;"> BookRepository : IBookRepository<br /> {<br /> </span><span style="color:#008000;">//</span><span style="color:#008000;">Snip</span><span style="color:#008000;"><br /></span><span style="color:#000000;"><br /> </span><span style="color:#0000FF;">public</span><span style="color:#000000;"> IList</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;"> GetFirstBookWithTitle(</span><span style="color:#0000FF;">string</span><span style="color:#000000;"> title)<br /> {<br /> DetachedCriteria crits </span><span style="color:#000000;">=</span><span style="color:#000000;"> DetachedCriteria.For(</span><span style="color:#0000FF;">typeof</span><span style="color:#000000;">(Book));<br /> crits.Add(Expression.Eq(</span><span style="color:#800000;">"</span><span style="color:#800000;">Title</span><span style="color:#800000;">"</span><span style="color:#000000;">, title));<br /> </span><span style="color:#0000FF;">return</span><span style="color:#000000;"> ActiveRecordMediator</span><span style="color:#000000;"><</span><span style="color:#000000;">Book</span><span style="color:#000000;">></span><span style="color:#000000;">.FindFirst(crits);<br /> }<br /><br /> </span><span style="color:#008000;">//</span><span style="color:#008000;">Snip</span><span style="color:#008000;"><br /></span><span style="color:#000000;"> }</span></div></pre><!-- Code inserted with Steve Dunn's Windows Live Writer Code Formatter Plugin. http://dunnhq.com --></div><p>This is a specific example of a general problem - making the application do work it doesn't need to do.</p><p>I'd generally be very suspicious of repositories which return unfiltered lists of any big aggregates. They can be very useful for test purposes, but can really kill you if they get into the production code. Keep an eye out for filtering being performed in C# code which would be simpler and more efficient as a relational database query. These tasks should be pushed down into your repository classes and rewritten in HQL or Criteria queries.</p><p><strong><strong>Avoidable </strong>Mistake - Loading in data that you don't need</strong></p><p>Bear in mind that calls to the database are expensive, especially if said database is on a remote machine. Every time you go to the database for something that you don't need, you are going to slow down your app and needlessly waste resources. Here are some examples of mistakes we made:</p><ul> <li>Loading objects from the database to perform unnecessary validation.</li> <li>Eagerly loading objects on page load "just in case" the app might use them.</li> <li>Repeated loads of objects from the database due to page lifecycle events.</li> <li>Loading aggregates when a report query or a summary object would be better</li></ul><p><strong>Avoidable Mistake - Forgetting to check the SQL being generated</strong></p><p>Make sure you check the SQL statements being used by NHibernate to satisfy your requests. Sometimes you will get quite a surprise. To enable logging of the SQL statements to the console when running your tests, add </p><div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:50421c7e-a177-4cbd-8d49-781dc3024429" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;;overflow: auto;"><div><!-- Code highlighting produced by Actipro CodeHighlighter (freeware) http://www.CodeHighlighter.com/ --><span style="color:#000000;"><</span><span style="color:#000000;">add key</span><span style="color:#000000;">=</span><span style="color:#800000;">"</span><span style="color:#800000;">show_sql</span><span style="color:#800000;">"</span><span style="color:#000000;"> value</span><span style="color:#000000;">=</span><span style="color:#800000;">"</span><span style="color:#800000;">true</span><span style="color:#800000;">"</span><span style="color:#000000;"> </span><span style="color:#000000;">/></span></div></pre></div><p> to the activerecord section in your app.config file . If you want to log application output, Ken Egozi has a post showing you how to do it <a href="http://www.kenegozi.com/Blog/2008/04/03/logging-sql-output-from-nhibernate-using-log4net.aspx">here</a>.</p>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com4tag:blogger.com,1999:blog-30913342.post-85179627368572714442008-09-05T08:38:00.003+01:002008-09-05T08:54:47.621+01:00Dogfooding NSynthesis<p>Earlier this year <a href="http://www.nutrun.com/">George</a> and I came up the with idea of trying to join up the mock interactions in our unit tests to improve the confidence that our tests were coherent and that all our mocked interactions hung together. The result of this was <a href="http://synthesis.rubyforge.org/">Synthesis</a>. </p> <p>The only problem was that I'm working on .NET projects at the moment. Enter NSynthesis. </p> <p>The aim of NSynthesis is to provide 'code coverage for mocked interactions for .NET'. It uses PostSharp to detect mock expectations being set by your test code and verifies that there is a test of the real production implementation in the same test run. If this call is missing, we fail the build. Alex and Bernardo and I are busy working on getting NSynthesis to the point where we can have a 0.0.1 release and it is looking quite promising now. Out of curiosity, I thought I'd grab the trunk build and try to run NSynthesis on a subset of my current project.</p> <p>I disabled the generic support as <a href="http://alexscordellis.blogspot.com/2008/08/synthesised-testing-and-net-generics.html">we are still working on this</a> and it is not really stable yet. I also filtered the classes which we are using to only include the our services namespace. Services should be easy to unit test, as there is always a repository or another service between them and the real world. We also find this is where we have the most mocked interactions. </p> <p>The results. 54 unit tests analysed, 12 unique mock calls made, 6 unique untested mock calls! <br />That's 50% of the mock interactions not being unit tested! Blimey!</p> <p>So. There is definitely value in using this tool! It is not that this code is untested - there are lots of wired tests and acceptance tests. However, what it highlighted was that we could have tested a lot more code at a lower level, and possibly would have sped up our test suite as a result. It was a surprise for the team too - we all thought we were already unit testing everywhere we could. </p> <p>When I extended NSynthesis to run against the whole code base, I ran into another problem. What broke NSynthesis was the situation where a class 'acquires' an implementation of an interface. </p> <p><a href="http://lh6.ggpht.com/stooferus/SMDh7c_shKI/AAAAAAAAACM/3nQ3JJ4Cws0/s1600-h/image%5B10%5D.png"><img height="176" alt="image" src="http://lh4.ggpht.com/stooferus/SMDh8ACLcSI/AAAAAAAAACU/0TCUzaeUD80/image_thumb%5B6%5D.png?imgmax=800" width="650" /></a> </p> <p>Here, my repository does not actually implement Exists itself, rather it derives from an existing implementation. In the code, this was our own BaseRepository, which in turn inherited from Castle's ARRepository<T>. Now when I mock the repository, I mock the interface. When I test my repository, I'm going to test the Repository class itself. NSynthesis then completely failed to realise that I have a tested implementation for IRepository because it is looking for a concrete implementation by a class which implements the interface IRepository. And BaseRepository does not. </p> <p>I think getting this working could be fun :-)</p> Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com2tag:blogger.com,1999:blog-30913342.post-6316521747298642652008-04-01T08:16:00.003+01:002008-04-01T09:04:36.479+01:00Synthesis comes to ManchesterFollowing on from our presentation to <a href="http://www.euruko2008.org/">EuRuKo 2008</a>, George and I will be speaking at the next <a href="http://www.thoughtworks.com/what-we-say/events/geek-nights-man_uk.html">Thoughtworks Manchester Geek Night</a> on the 8th April. <br /><br />I'm an alumnus of UMIST - so I'm really looking forward to seeing the old place again, it has been a long time since I was last in Manchester!Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-34580674141860397402008-03-22T18:14:00.008+00:002008-03-23T16:42:04.644+00:00Synthesis and test confidence<a href="http://www.nutrun.com/">George</a> and I spoke to a bunch of friends about <a href="http://synthesis.rubyforge.org/">Synthesis</a> a few weeks ago at the inaugural Reading Geek Night. Around six of us met in a pub in Reading and got really confused looks from the punters looking on on us as we waxed enthusiastically about TDD concepts and how we thought that they could be improved. I found it very interesting to get feedback from a group of really smart non ThoughtWorkers . They don't all practice “full on agile” but they certainly do understand what it means to build and ship working code, on time, with tight business pressures. It has taken a couple of weeks to digest the feedback, hence the slow time to post about the event.<br /><br />So, here is another attempt to explain why we may need synthesis on our projects in the light of Reading Geek Night #1 and previous discussions with the guys at ThoughtWorks UK previous to this....<br /><br />If you practice TDD, then the chances are that you already have a large number of unit tests. You may have a bunch of other automated tests of different types as well (functional, integration, performance...), and if you do, that's great. Keep doing that.<br /><br />At the other end of the spectrum, you will also have some form of acceptance testing structure in place. The format of this varies from project to project. It could be a set of manual scripts which your Testing/QA team/Nominated Guy run, or it could be a bunch of system tests using FIT or one of the BDD frameworks. Maybe you paid for a commercial product and have something like a bunch of Rational Robot scripts.<br /><br />We currently have a large disconnect between our high level tests and our low level unit tests, especially when you consider how confident you would be that the system works as desired by running one or more if the different types of test in the system.<br /><br />At the bottom of the scale there is a system which has no tests. I would not be at all confident that this system worked. At the top end of the scale, I have a system which is running live in the real production environment and is being used by the actual user base. I am very confident that this system is working. At some point in the past, most businesses came to the conclusion that IT cannot blindly put systems live to discover if they work, hence all the interest in testing techniques.<br /><br />High level system tests give us much more confidence that a system may work in production than unit tests because of the simple fact that they are exercising the production code in a more realistic manner; the data is more real, and all the interactions between the components are 'real', or as close to real as we can get in a test environment.<br /><br />Unit tests provide a much weaker level of confidence due a number of issues when you want to use them to prove that a system works.<br /><br /><span style="font-weight: bold;">Unit tests are incomplete</span><br />We cannot unit test all of our code. Note that good design practices can massively reduce the amount of 'untestable' code in the system and by practicing TDD and running code coverage tools it is simple to provide a view of how well we are doing. However, there are always parts of the system which you either cannot test or neglect to unit test for some reason.<br /><br /><span style="font-weight: bold;">Unit tests are disconnected from other unit tests</span><br />Unit tests test a small amount of code, by definition, and often in isolation. If you are testing interactions between components then these are simulated using mocks. How can I be confident that I got my mock interactions correct? Also, how can I be confident that I have tested or even completed coding up the real version of whatever it is I am mocking? Again, we can mitigate this by reducing the number of interaction based unit tests and by writing state based unit tests when we can. However sometimes it is much more natural to follow the interaction test approach, and this incurs a risk that we are not simulating our interactions accurately.<br /><br />Please don't take away from this that unit tests are bad. Unit tests are the lifeblood of a healthy project and are great for proving that the individual cogs are properly machined. What they do not tell you is whether you have the right number of cogs, or if they all fit together properly. If I want to know that components A,B, and C really play well together, then I need to write another aggregate test which puts ABC together and validates the unit tests got the interactions correct in a more realistic environment. George and I would call these functional tests, but you may use another term on your project. You may not have these tests on your project either – and that could be more worrying still – this means there could be gaps in your coverage at the levels that developers work at.<br /><br />So, if I have functional tests plus unit tests, I am more confident that my system is working before I commit to running the slow system tests. B y running functional tests I have confidence that I have wired up all my well behaved units of code in a meaningful manner and that they are playing well with each other as I predicted. The chances are, that the wider system is going to work. I still am not as confident that the system works as I will be when my acceptance test suite is run by [insert machine/human here], but I can probably sleep well enough.<br /><br />Now, some functional tests are inescapable and add a view into an aspect of the system that a unit test just cannot provide. A great example if you are using Spring would be to prove that you can get your components from the container and that they are wired up correctly. However, if you are purely proving that your simulated interactions are valid in the unit tests, then repetition is creeping into the process - which is wasteful.<br /><br />This is where Synthesis comes in. Synthesis monitors the simulated interactions which you create in your unit tests and verifies that there is a corresponding unit test that exercises and validates the real object in your system. If everything joins up, then your build passes. If there are disconnects, then the build fails. So, if I have a unit tests for A, B, and C, then it is fine for me to simulate A's interactions with B and C using mocks, providing synthesis can match all my expectations with real tests which make the very same calls to the real A. The result is a synthetic bigger test, where A, B, and C are virtually linked by their expectations.<br /><br />Obviously, there is a sliding scale of 'matching'. At one end you have basic method name matching, and at the other end of the scale there is full data matching for every call. The closer you get to true data matching, the more confidence is gained, but the more restrictions are placed on developers. We are not sure how far we need to go in order for a team to get an optimal balance between speed and safety – but I'd be very interested to get opinions on this matter :-)<br /><br />Currently, Synthesis validates the call signature completely, but does not check the content of data passing between the mocks and data used to test an object for real. We plan to add this soon. However, there is a benefit to be had by ensuring that all your interactions join up. This will catch method mismatches, dynamic calls which are not tested such as Active Record queries, and gaps in your test coverage which have simply been missed by those fallible human developers!Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com3tag:blogger.com,1999:blog-30913342.post-36508258891025041472008-01-05T18:00:00.000+00:002008-01-05T21:05:55.586+00:00Dear MicrosoftI thought it was very sporting of you to somehow break your MSDN subscriber downloads page for your flagship IE7 web browser:<br /><br /><a href="http://www.flickr.com/photos/34995768@N00/2169585324/" title="Broken Download by Stoofer, on Flickr"><img src="http://farm3.static.flickr.com/2103/2169585324_4b8b00c11a.jpg" width="500" height="375" alt="Broken Download" /></a><br /><br />But not for Firefox:<br /><br /><a href="http://www.flickr.com/photos/34995768@N00/2168790837/" title="Firefox ok by Stoofer, on Flickr"><img src="http://farm3.static.flickr.com/2362/2168790837_cc4999c048.jpg" width="500" height="375" alt="Firefox ok" /></a><br /><br />Well done! I didn't even realize that the site had any problems at all until I switched over to my virtual machine of XP from Ubuntu in order to download the ISO I needed via your custom file transfer application. Why do you insist that I use this at all? It seems really odd.<br /><br />Anyway, perhaps I should just try to get your File Transfer Manager to run in Wine and I could have a seamless experience browsing your web site from a proper OS....<br /><br />Love and hugs,<br /><br />StuStooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-65808090090753362712007-02-11T23:32:00.000+00:002007-02-11T23:32:11.744+00:00Damn SkippyFrom a conversation whilst pairing with a client developer as we attempted to test infect some legacy code last week.<br /><br /><span style="font-style:italic;">Me:</span> Phew! Well, that little bit is under test now. We can start to fix it up now.<br /><br /><span style="font-style:italic;">Client Dev:</span> That seemed too hard....you know, if the guys who wrote that had had to put in tests as they went, the API would NEVER have looked like this. It would just have been too painful to work with.<br /><br /><br />It really is nice to witness the penny drop. True job satisfaction.Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com1tag:blogger.com,1999:blog-30913342.post-7729947976780429072007-02-05T21:03:00.000+00:002007-02-05T21:04:28.452+00:00Test Code and Production Code – two distinct beastiesThere are some common qualities that production and test code must possess in order to enhance their status as ‘good code’. Typical dimensions which one could apply may be conciseness, clarity of code, performance, elegance and extensibility (the list goes on). These are all good methods of assessing ‘good code’. However what differentiates good test code from good production code is not so clear when only these traditional qualities are taken into account. It may be more useful to consider what the drivers for the two type of code are.<br /><br />For production code, these traditional values are useful, but cannot be assessed meaningfully if the code does not fulfil a business requirement. Ultimately this is the only driver of production code – it has to get the job done for the business. If your widget selling app for Widgets R Us cannot help to sell widgets, then no one will really care how speedy, elegant and extensible your Ultra Widget Framework is. The business code also needs to be robust enough for the business demands to allow the application to stay up and running cheaply enough for them to get a return on the build. It is a harsh world we live in and production code has to cut it in an unjust world.<br /><br />So what about the test code? Test code has a different set of drives which compliment the production code. The primary driver is to assist in delivering high quality production code which robustly meets the requirements of our client. To this end the code also needs to drive the design and document the expected behaviour of our production code. <br /><br />Driving the design of the code pushes the code towards delivery of production requirements, but this is only part of the story. Test code can used to accurately model the expected behaviour of the system and ensure that all key behaviour aspects of the system really do behave as per the requirements. By driving the design of the code with tests the system can meet the key delivery and quality drives of the code, but allows the developers the opportunity to make other improvements which improve the traditional values of the code. For example, a developer is more likely to refactor towards an elegant pattern which is emerging when a safety net in place than without one. If our tests measure performance then we are more likely to improve our performance qualities.<br /><br /> The test code is also a driver to document the behaviour accurately and clearly in order to provide a suitable level of traceability to the team and the stakeholders to demonstrate that the business need really have been met. When the test code is clear and comprehensive then the application benefits from not only the traditional test safety net, but also executable documentation that can highlight where and how the application has veered from the course of the business requirements.<br /><br />So do traditional values not matter any more? Not at all - these values are crucial to quality code – in test and in production. However, you may assess the values differently in test and production code based when you consider what the code is driving to achieve. What may be reasonable in production code may be unsuitable for test code, as the application of a particular approach may obstruct the drives of test code to drive the design of the production code or clouds the documentation qualities of the test code by making a test less clear.<br /><br />Put another way, just like in acting, it is important to consider your code’s motives first and then apply the traditional values in the context of what the code is trying to achieve.Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-50349698644446812222006-11-14T19:53:00.000+00:002006-11-14T20:45:58.268+00:00Where there's no sense, there's no feelingI finally got a chance to go beating again on Saturday. I help out as a beater at a couple of the local pheasant shoots; a good day's beating is about as much fun as you can have in the countryside with your trousers on!<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://static.flickr.com/12/69106518_7bf7247394.jpg"><img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer; width: 500px;" src="http://static.flickr.com/12/69106518_7bf7247394.jpg" alt="" border="0" /></a>The season started over a month ago now and I have already been to one shoot. Unfortunately though I have been unable to take Lyra with me due to her suffering from 'lady dog problems'. Luckily for both of us this is all over and done with now. So we eagerly headed off to Ram Alley for a spot of pheasant chasing.<br /><br />As this was Lyra's first outing of the season, I was a little nervous about how she would perform, would there be any signs of canine resentment for being left behind on the previous shoots? Would the lack of training during those summer months I had spent away from home show at all?<br /><br />The first drive put all my fears to rest. I had to work Lyra through a patch of maize. This sort of cover crop can cause problems - dogs get very excited chasing through the cover looking for game (it probably feels like one of those jungle chase scenes in Predator to the dog) and the lack of visual contact can mean you loose control relatively easily. But she did really well - even retrieving a bird that had been overlooked by the pickers up :-)<br /><br />And so all through the day we went from good to better until we got home. Only then was it clear that my, by now dog tired, hound was finding it hard to settle down on the carpet. On closer inspection we realized that, although her paws and muzzle seemed completely immune to nettle rash, the inside of her ears were not. All inside both ear flaps she was read raw with nettle rash!<br /><br />Clearly she was having so much fun during the day that she hadn't noticed that her ears were glowing. I think I can understand though. Lyra is a working dog, and she is driven to work in the same way that many of the truly good developers I know are driven to code. Imagine a world where coding was only permitted September-January and you could only get access to a really useful computer 2 or 3 days a week at best. This is what it is probably like for Lyra (shudder). If this were the case, I don't think you would stop coding if you got a paper cut on a couple of fingers...<br /><br />Given the way the winter is going there will be nettles in the woods for a few weeks yet and there's no way I can leave the dog behind. So, now I'm surfing the net for nettle rash cures for Weimaraners...Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-38226315058644795202006-11-14T17:04:00.001+00:002006-11-14T17:41:18.859+00:00Is the clutter in your test trying to tell you something?We’ve all encountered it at some point or another. Your unit test is doing the job, but the test is so bogged down with setup code that it is impossible to see the wood for the trees. The interesting test code is in there somewhere, but where exactly?<br /><br />The common causes of clogs in your test code are:<br /><ul><li>Complex object creation code. </li><li>Preceding interactions with the test subject.</li></ul><br />Complex object creation is simple to deal with. You can create test fixtures to create the object for you, or better still you can refactor the code to try and simplify things a little.<br /><br />But what about the interaction clutter? This is more subtle in the way that it insinuates itself into the test code. The clutter sneaks up on you. It demands more and more code to be added due to 'unavoidable' interactions with one object after another. Soon the useful to useless LOC ratio in the test case has tipped. Suddenly it becomes the norm that you should have N interactions in the test case to simulate calls to the database and N more calls out to other supporting services, even though the aspect of the code which you want to test has no real need to go through these extra steps.<br /><br />For a particularly contrived example, lets imagine that we are building an online Pie ordering service for Weebl & Bob's Pie Delivery Service…and we are implementing it using WebWork.<br /><br />We are in the final stages of implementing our pie ordering action. We want to verify that on execution the action invokes a number of support services:<br /><br /><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">public</span> <span class="Type">void</span> testShouldOrderBobAYummyPieOnSubmit() {<br /> PieOrder yummyPieOrder = <span class="Keyword">new</span> PieOrder(“yummypie”, 1, “<span class="Statement">for</span> Bob”);<br /> PieOrderingAction pieAction = <span class="Keyword">new</span> PieOrderingAction(orderService);<br /> pieAction.setAction(Action.SUBMIT);<br /> pieAction.execute();<br /> assertEquals(yummyPieOrder, orderService.getLastPieOrdered())<br />}<br /></pre></td></tr></tbody></table><br />But our test fails to run - it never reaches our orderService throwing nasty null object exceptions instead. But why? Looking more carefully at our execute method we make a number of discoveries:<br /><ol><li>When our action executes, it actually loads our order from the database using its repository and the supplied pie-ID. This test doesn’t supply a repository or a pie-ID. So we buckle down and add it into the test.</li><li>Weebl and Bob’s Pie Delivery Service often runs out of stock. So before we can allow our customer to order our pie we need to call out to our pieInventoryService to ensure that the yummy pie actually exists and is ready to be cooked for our customer. Grrrr. We add more support code into the test.<br /></li></ol>Now our test has become a bit of a monster:<br /><table class="code"><tr><td><pre><br /><span class="Modifier">public</span> <span class="Type">void</span> testShouldOrderBobAYummyPieOnSubmit () {<br /> PieOrder yummyPieOrder = <span class="Keyword">new</span> PieOrder(“yummypie”, 1, “<span class="Statement">for</span> Bob”);<br /> PieOrderingAction pieAction = <span class="Keyword">new</span> PieOrderingAction(orderService);<br /> pieAction.setPieOrderRepository(pieOrderRepositoryMock);<br /> pieOrderRepository.expects(once()).method(loadOrder)<br /> .will(returnValue(yummyPieOrder));<br /> pieAction.setPieInventoryService(pieAlwaysInStockStub);<br /> pieAction.setAction(Action.SUBMIT);<br /> pieAction.setPieID(yummyPieOrder.getPieD());<br /><br /> pieAction.execute();<br /><br /> assertEquals(yummyPie, orderService.getLastPieOrdered())<br />}<br /></pre></td></tr></table>OK, so this isn’t the end of the world. But it isn’t so hot either in pastry-free, real world scenarios, this could be much worse. The problem here is that our implementation of execute has 3 phases:<br /><ol><li>Load PieOrder</li><li>Verify PieOrder</li><li>Submit PieOrder<br /></li></ol><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">public</span> <span class="Type">class</span> PieOrderPacingAction…{<br /> <span class="Modifier">public</span> <span class="Type">void</span> execute(){<br /> <span class="InlineComment">//Load pie order…</span><br /> <span class="InlineComment">//validate pie order…</span><br /> <span class="InlineComment">//submit pie order…</span><br /> }<br />}<br /></pre></td></tr></tbody></table>We really only want to test phase 3 in isolation. Someone has already implemented tests and production code for steps 1 and 2 – we don’t want to repeat ourselves.<br /><br />Time to listen to the test. Our test code is suggesting that this is not an optimal implementation. We could choose to hide all this away in obscure setup code, or we could try to find a better way of implementing our code. So we pick the best option and look into the API of our actions and discover that we could implement this differently. WebWork supports actions which implement <span style="font-family:courier new;">prepare()</span> and validate() methods. Woot! We can move our code into these calls instead and simply our production code.<br /><br /><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">public</span> <span class="Type">class</span> PieOrderPacingAction…{<br /> <span class="Modifier">public</span> <span class="Type">void</span> prepare(){<br /> <span class="InlineComment">//Load pie order…</span><br /> }<br /> <span class="Modifier">public</span> <span class="Type">void</span> validate (){<br /> <span class="InlineComment">//validate pie order…</span><br /> }<br /> <span class="Modifier">public</span> <span class="Type">void</span> execute(){<br /> <span class="InlineComment">//submit pie order…</span><br /> }<br />}<br /></pre></td></tr></tbody></table><br /><br />The beauty of this is that, in theory we can chop up our tests.<br /><ol><li>Our first set of tests check that the action loads the PieOrder during prepare().</li><li>Our second set of tests prove that the action validates its order during validate()<br /><span style="font-weight: bold; font-style: italic;">Given that the action has already loaded the order..</span></li><li>Our third set of tests verify that the action will submit the pie order during execute()<br /><span style="font-weight: bold; font-style: italic;">Given that the action has already loaded and validated the order.</span></li></ol><br />So our test case looks like this now:<br /><br /><table class="code"><tbody><tr><td><pre><br /><br /><span class="Modifier">public</span> <span class="Type">void</span> testShouldOrderBobAYummyPieOnSubmit () {<br /> PieOrder yummyPieOrder = <span class="Keyword">new</span> PieOrder(“yummypie”, 1, “<span class="Statement">for</span> Bob”);<br /> PieOrderingAction pieAction = <span class="Keyword">new</span> PieOrderingAction(orderService);<br /> TestUtils.setPrivateField(pieAction, “pieOrder”, yummyPieOrder);<br /> pieAction.setAction(Action.SUBMIT);<br /><br /> pieAction.execute();<br /><br /> assertEquals(yummyPie, orderService.getLastPieOrdered())<br />}<br /></pre></td></tr></tbody></table><br /><br />Those following closely will have noticed a call to set the pieOrder field via reflection. So why did we set the private field here (using evil reflection of all things!)? Well, this comes back to not <a href="http://nutrun.com/weblog/the-testing-anti-patterns-drafts-vol-1/">test driving the code the wrong way</a>. We could have added a setter to allow a more traditional way of setting the object state. However, this is not useful to the real world. We are only setting the internal field in order to get our object into the correct state to test it. It is less evil to use reflection to put an object into a valid state for a test situation than it is to add access to internal state legally for all code.<br /><br />So what are the issues that we have to watch out for here?<br /><ul><li>Obviously the test is now more tightly coupled to the implementation details of our test subject. If we change the internal structure of our object, we could well break our tests. We can mitigate this by not making our object complicated. In this example we only reach into our object to set one field. Try and keep things this simple when using this technique</li></ul><ul><li>There is a risk that the tests don’t join up properly. If you are going to reach into an object to set its state, you must have corroborating test cases which prove that the object is in fact in this state at the end of the preceding calls in its lifecycle. It may also be worth writing a small number of ‘integration tests’ which run the object through the whole call cycle (here, prepare -> validate -> execute) in order to demonstrate the expected behaviour of caller and to prove that you got things right with the lower level testing.</li></ul>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com2tag:blogger.com,1999:blog-30913342.post-18218536904045807802006-10-19T14:29:00.000+01:002006-10-26T15:46:18.593+01:00Buried Intent. Bad for campers, bad for TDDRemember that your test code is not production code. The purpose of your test code is to prove that your production code works and to provide live documentation. Qualities which make good test and production code are not necessarily the same. For example, it is important that a test case clearly demonstrates the expectations of a test case inline. This is especially true when working on an XP project, as these tests are your primary means of documenting the expected behaviour of your test classes. If another developer cannot read your tests easily and determine what was intended of the production code then trouble will ensue<br /><br />Consider the following test case:<br /><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">public</span> <span class="Type">void</span> testValidateOffPeakUserConnectionRejectedDuringPeakTimes() {<br /> ClientConnectionRequest connectRequest = <span class="Keyword">new</span> ConnectionRequest(<span class="String">"bob"</span>, <span class="String">"12345"</span>);<br /> setMonitorExpectations(<span class="String">"bob"</span>, <span class="String">"Some account"</span>, <span class="String">"12345"</span>, ON_PEAK, <span class="Keyword">true</span>, <span class="Keyword">false</span>, <span class="Keyword">true</span>);<br /><br /><span class="Type"> boolean</span> requestResult = connectionMonitor.validate(connectRequest);<br /> assertEquals(<span class="Keyword">false</span>, requestResult);<br />}<br /></pre></td></tr></tbody></table><br />Superficially, this test seems to be OK. The test is short and sweet, and we are checking that our monitor rejects the request. But is going on exactly? Well, to work this out we have to dig down into the <code>setMonitorExpectations</code> method. More information is revealed:<br /><br /><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">private</span> <span class="Type">void</span> setMonitorExpectations(<br /> String clientName,<br /> String accountName,<br /> String clientId,<br /><span class="Type"> boolean</span> introductory,<br /><span class="Type"> boolean</span> unlimited,<br /><span class="Type"> boolean</span> offPeakOnly) {<br /><br /> UserAccount account = <span class="Keyword">new</span> UserAccount(12345L);<br /> account.setAccountName(accountName);<br /> account.setClientName(clientName);<br /><br /> buildAccountType(introductory, account);<br /><br /> repositoryMock.expects(atLeastOnce()).method(<span class="String">"find"</span>).will(returnValue(account));<br /> repositoryMock.expects(once()).method(<span class="String">"isUnlimited"</span>).will(returnValue(unlimited));<br /><br /><span class="Statement"> if</span> (offPeakOnly) {<br /><span><span class="Statement"> </span></span><span><span class="Statement"> </span></span>account.setAccountRestrictions(AccountUsage.RESTRICTED);<br /><span><span class="Statement"> </span></span><span><span class="Statement"> </span></span> repositoryMock.expects(once()).method(<span class="String">"getAllowedConnectPeriods"</span>)<br /><span><span class="Statement"> </span></span><span><span class="Statement"> </span></span> .with(eq(12345L),eq(clientId)).will(returnValue(offPeakOnly));<br /><span><span class="Statement"> </span></span>}<br />}<br /></pre></td></tr></tbody></table><br />Oh, so we are setting up a bunch of attributes of the account, conditionally flagging the account as restricted, and then going on to add an expectation. But we still don't have the full picture, as there is yet another call out to <code>buildAccountType</code>.<br /><br /><table class="code"><tbody><tr><td><pre><br /><span class="Modifier">private</span> <span class="Type">void</span> buildAccountType(<span class="Type">boolean</span> introductory, UserAccount account) {<br /><span><span class="Statement"> </span></span><span class="Statement">if</span>(introductory) {<br /><span><span class="Statement"> </span></span>account.setAccountType(AccountType.INTRODUCTORY);<br /><span><span class="Statement"> </span></span>}<br />}<br /></pre></td></tr></tbody></table><br />Now we have a a number of problems.<br /><ul><li>Although the test code works, and is compact it is failing to fulfil the role of documenting the production code clearly. In order to determine what my test methohd is trying to prove I have to dig down a number of levels.<br /></li><li>I suspect that the helper method is actually too rich for this method. There is conditional logic in there to change the expected test behaviour based on the test situation. This may be acceptable in production code, but conditional logic has no place in test code.</li><li>The fact that the interactions with other objects are buried in the helper functions is also hiding potential issues in the production code. In this case it is the repeated calls to the repository. By inlining the expectations it would be more obvious that the classes have a chatty relationship, and this could be harnessed by changing the test expectations and driving the production code to become more efficient.<br /></li></ul>This is a natural trap which we can fall into when we are constructing unit tests. Our IDE allows us to extract methods so easily these days that we often find that we are extracting "common code" from our test calls because we believe that this is helping to keep the code base compact and efficient. In fact, this naive refactoring can case real problems. Now that our test code is less clear to a casual reader, the overhead of maintaining the test suite could rise!<br /><br /><span style="font-size:130%;">Ways Out Of this Situation?</span><br />In this situation the simplest route out is to inline all the buried methods and step back. In this case we initially end up with:<br /><br /><table class="code"><tr><td><pre><br /><span class="Modifier">public</span> <span class="ValueType">void</span> testValidateOffPeakUserConnectionRejectedDuringPeakTimes() {<br /> ClientConnectionRequest connectRequest = <span class="Keyword">new</span> ConnectionRequest(<span class="String">"bob"</span>, <span class="String">"12345"</span>);<br /><br /> UserAccount account = <span class="Keyword">new</span> UserAccount(12345L);<br /> account.setAccountName(<span class="String">"Some account"</span>);<br /> account.setClientName(<span class="String">"bob"</span>);<br /><br /> <span class="Statement">if</span> (ON_PEAK) {<br /> account.setAccountType(AccountType.INTRODUCTORY);<br /> }<br /><br /> repositoryMock.expects(atLeastOnce()).method(<span class="String">"find"</span>).will(returnValue(account));<br /> repositoryMock.expects(once()).method(<span class="String">"isUnlimited"</span>).will(returnValue(<span class="Keyword">true</span>));<br /><br /> <span class="Statement">if</span> (<span class="Keyword">true</span>) {<br /> account.setAccountRestrictions(AccountUsage.RESTRICTED);<br /> repositoryMock.expects(once()).method(<span class="String">"getAllowedConnectPeriods"</span>)<br /> .with(eq(12345L), eq(<span class="String">"12345"</span>)).will(returnValue(<span class="Keyword">true</span>));<br /> }<br /><br /> boolean requestResult = connectionMonitor.validate(connectRequest);<br /> assertEquals(<span class="Keyword">false</span>, requestResult);<br />}<br /></pre></td></tr></table><br /><br />This is clearer than we had before, and when the method is rearranged and the account creation logic extracted into a helper method we end up with:<br /><br /><table class="code"><tr><td><pre><br /><span class="Modifier">public</span> <span class="ValueType">void</span> testValidateOffPeakUserConnectionRejectedDuringPeakTimes() {<br /> UserAccount account = buildAccount(<br /> 12345L, <span class="String">"Some account name"</span>, <span class="String">"bob client"</span>, INTRODUCTORY, RESTRICTED);<br /> ClientConnectionRequest connectRequest = <br /> <span class="Keyword">new</span> ConnectionRequest(account.getClientName, account.getId());<br /><br /> repositoryMock.expects(atLeastOnce()).method(<span class="String">"find"</span>).will(returnValue(account));<br /> repositoryMock.expects(once()).method(<span class="String">"isUnlimited"</span>).will(returnValue(<span class="Keyword">true</span>)); <br /> repositoryMock.expects(once()).method(<span class="String">"getAllowedConnectPeriods"</span>)<br /> .with(eq(12345L), eq(<span class="String">"12345"</span>)).will(returnValue(<span class="Keyword">true</span>));<br /> <br /> boolean requestResult = connectionMonitor.validate(connectRequest);<br /> assertEquals(<span class="Keyword">false</span>, requestResult);<br />}<br /></pre></td></tr></table><br /><br />Note that this refactored test is really not much larger than the original. However the difference is that the test expectations are clearly visible. As a result, the developer is more likely to be in a position to improve the code. Here the mock object interactions are showing that we have an overly chatty relationship with another object. This could be a sign that we are suffering from the 'feature-envy' anti pattern in our production code, which in this example could lead to us making bitty calls to the database and could become a performance bottleneck. By exposing the relationship explicitly in the test case, we can now choose what to do about the situation.<br /><ul><li>Live with the problem - this is not an important feature<br /></li><li>Change the expectations to drive improvements in the production code. It would now be easy to change the test case to forbid the look up calls to the repository and drive the production code down a more performant path.<br /></li><li>Break up the method under test.</li><br /></ul>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com2tag:blogger.com,1999:blog-30913342.post-7937915741012261822006-09-16T12:04:00.000+01:002006-09-16T12:06:17.650+01:00The Testing Anti-Patterns Drafts Vol. 1 (draft2)<a href="http://nutrun.com/">George</a> and I both consider ourselves fortunate enough to work as members of teams where TDD is practiced by default.<br /><br />There are two reasons beyond the obvious as to why we find it relatively harder to code in a non TDD approach. Test Driving our code enhances our vision on how to design/model/instrument the application's universe. Also, starting with a test means work begins and ends with coding, not meetings, discussions or modelling our vision in pictures bound to be proven unrealistic when the first coding bottlenecks arise.<br /><br />This is not to say that discussions or modelling are intrinsically bad things, a good brisk white board discussion can really help – but don't ever forget that this is a relatively abstract activity.<br /><br />Tests have a much closer relationship with your code that allows you to discover and document the behaviour of your system in a much more detailed manner. In fact it is the close relationship between tests and code that can lead to problems. This relationship must be kept in balance if the process is to be successful.<br /><br />Production code is the bread winner. This is where your business value lies. However, due to the nature of the TDD process, the production code is also somewhat dim-witted; it doesn't really have a clear vision or motivation in life. The test code is there to explain to the code what is expected of it and to highlight any mistakes the code makes by identifying in a precise manner why the code doesn't quite do the right thing and to point out how to head off in the correct direction. In some respects this could be likened to the relationship between a boxer and his trainer. Ultimately, it is the boxer who has to go into the ring and win the fight, but it is the trainer who gets the boxer into the zone mentally and physically.<br /><br />Another equally important role of the test code is to document the expected behaviour of the production system. This is fantastic! Suddenly, the technical documentation is no longer a static and dusty tome on a shelf, rather an ever accurate and clear guide to what the code really does. If your code is not fit for purpose, this should be made glaringly obvious in your tests, and by changing your documentation you should be able to change the behaviour of your code to make it do the right thing in the right way.<br /><br />However, we don't live in a perfect world and testing properly is not easy, despite the presence of opposable thumbs and shiny new Macs. Once you get past the simplistic examples and into the real world you find that it is difficult to write effective tests, and often the tests are much harder to craft than the actual code. In many ways, this should not come as a surprise. When you write a test, you are attempting to satisfy a number of aspects of the system: quality, design, documentation. And all of this expressed though the medium of a software language!<br /><br />So, what makes our tests go bad? There seem to be a number of patterns starting to emerge, but to list them all here is going to take too long – I think George and I are going to have to break this out (hopefully with lots of assistance from our friends ;-)). In my mind the coarse categories of TDD anti patterns are:<br /><ol><li>Driving the code the wrong way.</li><li>Hiding deficiencies in the code.</li><li>General test smells<br /></li></ol><h3>Test Driving The Wrong Way</h3>This is the situation where your code starts to bend towards the test code to meet requirements of the <span style="font-style: italic;">testing framework </span>rather than to meet the needs of the <span style="font-style: italic;">application</span>. Examples of this include:<br /><ul><li>Adding calls to your production data layer code to allow setup/teardown/query operations which are not required by the application.</li><br /><li>Providing access to properties which should not be made visible normally (this is what George is getting at with <a href="http://nutrun.com/weblog/the-testing-anti-patterns-drafts-vol-1/">Design Pervasive Testing</a>)</li><br /><li>Forcing the use of IOC where in fact it makes more sense to create objects internally or access static methods.<br /></li></ul><br />Clearly there is a sliding scale of smelliness here. Overuse of IOC is not really such a bad thing – the code will still function as required. It just leaves you thinking that there has to be a better way.<br /><br />Adding a deleteAllClients() method to your production code is a big deal though!<br /><br />Sometimes, this can be sign that the test framework that you are using to test your application is not suitable, as switching to a different technology can help get around the smell.<br /><br />For example - the use of DbUnit may make it easier to put your database into the correct state for a functional test and remove the need for the dodgy setup code in your data layer. Newer mocking libraries like JMockit can remove the absolute requirement for IOC based design patterns by allowing you to hook the creation of new objects and accessing static methods.<br /><br /><h3>Hiding deficiencies in the code<br /></h3>This is where we seem to be suffering the most. It is very easy to hide problems in the code base instead of actually driving them out with your tests. Examples of this phenomenon:<br /><ul><li>Hidden behaviour. The production code performs a variety of complex activities in a given situation. However this is not clear because the test has buried the behaviour in deep in a number of (often obscurely named) helper functions. This is often a sign that you have too many complex relationships between your objects, or that the conversations the component has with its collaborators is overly chatty.</li><br /><li>Supporting actors stealing the show. The test is so polluted with setup code that you can't actually make out what the test is trying to show you. This can often be a sign that the step is too complex.<br /></li></ul><h3>General Test Smells<br /></h3><ul><li>Badly named tests. This is especially bad when you consider the need to document the code through the medium of tests.</li><br /><li>Inappropriate use of stubs. For example, stubbing a simple data type.</li><br /><li>Etc.<br /></li></ul>In summary, I think we really need to care about the quality or our test code and learn to treat it quite differently to the production code, realising that the two portions of our code base serve different purposes in our development activities.<br /><br />So...over to George to fill in some more blanks ;-)<br /><br /><span style="font-size:85%;"><em>“The Testing Anti-Patterns Drafts”</em> is a collaborative effort between <a href="http://nutrun.com/">George “spring is overrated” Malamidis</a> and myself which aims to identify cases of <em>Testing gone bad</em>. It consists of a single document that will undergo constant enhancements and modifications, in a “pair-authoring” manner, utilising our respective weblogs as the platform. We hope to get input from anyone following the document, our goal being to produce an interesting resource for the TDD, or Testing Oriented in general community.</span>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com2tag:blogger.com,1999:blog-30913342.post-1157902902552836542006-09-10T16:16:00.000+01:002006-09-10T16:41:42.570+01:00Son, it's time to be a geek...This weekend a shiny new desktop arrived at the Caborn household. After many long months of procrastination, I finally bought a PC for my kids to use. The previous family PC had met an untimely end due to a spilt glass of orange juice about 6 months ago.<br /><br />While the kids were at school I spent a nice geeky morning setting it up for them and now Heather is happily playing preschooler games and Skyping to my laptop downstairs. Fantastic.<br /><br />My son wants to be a computer geek like his dad when he grows up. Well, not exactly like me; he plans to drive a DB7 and be a rock star Mondays and Fridays. So, if he's going to lead such a cool life, he needs to learn to program. I'd like to do this with him, in the same way that my dad tried to teach me how to be an engineer by getting out the Meccano set. PCs are easy to get to grips with and Phillip already understands how to put together a logical argument.<br /><br />At the age of six I think he may be ready to start to learn to program. The only question is: what language should we start with? I posed this question to a number of fellow ThoughtWorkers and, unsurprisingly they came back with a variety of answers!<br /><br /><ol><li>Lisp - "You should teach the guy a proper language and Lisp has everything in it"</li><li>Logo - "There nothing more rewarding than drawing a picture with a turtle and it made me what I am today!"</li><li>Flash - "Flash is easy to use and there's instant gratification - it is very visual"</li><li>Lego Mind Storms - "Its just so cool".<br /></li><li>Java - Java is easy to learn and I'm familiar with it.</li><li>Javascript - I can't remember why this was a good idea.</li></ol>So. I'm not certain, but I think I may go with Flash. I like the visual aspects of flash and most of the games and <a href="http://www.badgerbadgerbadger.com">silly things the kids love</a> on the net are made with flash. Wouldn't it be great to write a flash game together including scanned artwork provided by Heather (age 4)....<br /><br />Time to learn to do stuff with flash. I think I've got about a week before Phillip finishes his latest PS2 game and remembers that dad promised to teach him to code. Should be plenty of time to learn to do something cool in flash!Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com1tag:blogger.com,1999:blog-30913342.post-1154470372875842202006-08-01T23:09:00.000+01:002006-08-01T23:12:52.883+01:00It's tiddly, it's a wiki...It's <a href="http://www.tiddlywiki.com/">TiddlyWiki</a>!<br /><br />I'm playing with this at the minute as a replacement for the myriad of little emails, .txt files and other crud that I scatter across my laptop. It seems very nice and extremely user friendly.<br /><br />The appeal for me at the moment is that I can shove the whole lot onto a USB stick and keep it with me wherever I am.Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com0tag:blogger.com,1999:blog-30913342.post-1152536126727591252006-07-10T13:49:00.001+01:002008-11-13T23:27:02.483+00:00On graffiti and broken windows<p class="MsoNormal">In his book Tipping Point, Malcolm Gladwell describes how graffiti and broken windows can have a dramatic effect on the behaviour of the residents in a city. For those of you who have not read Tipping Point (and I strongly recommend you read the book), the key premises go something like this:</p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal"><b style="">Social change does not occur in the smooth linear way which many people imagine.</b><span style=""> </span></p> <p class="MsoNormal">Often social values will suddenly transition or ‘tip’ from one state into another.<span style=""> </span>Social change such as crime rates and fashion can often behave in a manner which is similar to the spread of diseases.</p> <p class="MsoNormal"><b style=""><o:p> </o:p></b></p> <p class="MsoNormal"><b style="">A tip can often be achieved by the compound effect of relatively benign factors.</b><span style=""><br /></span></p><p class="MsoNormal"><span style=""> T</span>oo much graffiti and too many broken windows can tip a neighbourhood from being a good neighbourhood into a crime ridden no-go area.<span style=""> </span>Additionally, the behaviour of key individuals in a social group can make quite bizarre actions (such as suicide) not just acceptable in a group, but also fashionable and desirable.</p> <p class="MsoNormal"><b style=""><o:p> </o:p></b></p> <p class="MsoNormal"><b style="">In order to repair a problem, you need to perpetually guard against seemingly insignificant factors in order to effect change.</b></p> <p class="MsoNormal">When the <st1:state st="on"><st1:place st="on">new york</st1:place></st1:state> underground decided to try to clean up the crime and vanadalism on the tube network, they started with the graffiti. The thoery was that by cleaning up the appearance of the trains the feel of the tube network would be improved and people would start to feel more secure and crime would be discouraged.<span style=""> </span>In order to clean up graffiti on their tube network they did it in a staged and defensive manner.<span style=""> </span>A small number of trains were designated as ‘clean trains’.<span style=""> </span>These trains were not allowed to slip and become covered in graffiti even though other trains remained ‘dirty’.<span style=""> </span>The number of clean trains was exteneded in a sustainable manner at the rate the tube system could cope with until the problem tipped and became controllable.</p> <p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal"><b style="">There goes the neighbourhood...<br /></b></p> <p class="MsoNormal">Applied to neighbourhoods, the principal is that an area can hit a point at which there are so many broken windows and walls covered in graffiti that it can change peoples social values. It suddenly becomes 'OK' to break more windows and deface property, and this can move on to more serious crimes being committed once people get a taste for misdemeanours. <span style=""> </span>At this point the neighbourhood has tipped and will rapidly go downhill.</p> <p class="MsoNormal">In order to affect change and tip a bad district back into the light, it is necessary to actively repair broken windows and clean up graffiti, because without improving the environment that people live in there will not be enough social impetus to allow the residents to control and discourage antisocial behaviour.<br /><!--[if !supportLineBreakNewLine]--><br /><!--[endif]--></p> <p class="MsoNormal"><b style="">Back in the world of software…</b></p><p class="MsoNormal"><br />It is interesting to consider if the concepts of the tipping point can be applied to software? I believe that they can. Many applications are perceived by their developers and maintainers to contain either 'good' or 'bad' code. Good code is much cherished by the teams who maintain them – and they bring joy and happiness to the world.<span style=""> </span>Bad code is a millstone around the necks of the maintenance teams and is painful to maintain.<span style=""> </span>But how do we judge good and bad code?<br /><br />A number of factors can come into play here, but the main dimensions that I feel developers and QA's tend to use to decide if code is good or bad are perceived design quality, and the number of defects in the code base. Poorly designed or ‘smelly’ code is the graffiti of software, and bugs are our broken windows.<br /><o:p> </o:p></p> <p class="MsoNormal"><b style="">Code Quality and graffiti<o:p></o:p></b></p> <p class="MsoNormal">Code quality is interesting.<span style=""> </span>There are a number of motivating factors which drive for high quality code.<span style=""> </span>Primarily it comes down to the values of the team.<span style=""> </span>If a team values good quality code, then it will attempt to write production code ‘the right way’.<span style=""> </span>However, in any project there are a number of competing drivers which can hamper the realisation of the quality code.<span style=""> </span></p> <p class="MsoNormal"><o:p> </o:p><br />In my mind the biggest anti-quality drivers are:</p> <ul style="margin-top: 0cm;" type="disc"><li class="MsoNormal" style="">The fact that there is a tight deadline.</li><li class="MsoNormal" style="">Members of the team who do not value quality code.</li><li class="MsoNormal" style="">Working in a bad district.</li></ul> <p class="MsoNormal">The first 2 of these anti quality drivers are common fare. But what about the bad code district?<span style=""> </span>Despite the desire of a team to write good code, they may well struggle if they find themselves in a bad neighbourhood. If the code contains too many code smells (there is a high level of graffiti and broken windows) you may well find that you are producing more smelly code.<span style=""> </span></p> <p class="MsoNormal"><o:p> </o:p><br />Why?<span style=""> </span>Because the developers have lost hope.<span style=""> </span></p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">The motivation to write good code and behave is much reduced if every preceding developer has treated the code so badly.<span style=""> </span>It becomes very easy for our developer to scrawl on the walls by writing a piece of smelly code or smash a few windows with the odd poorly handled exception because there are so many examples of this about.<span style=""> </span></p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">The <b style="">very same</b> developer would be much more likely to run up some scaffolding or lay down the dustsheets with the odd unit test in a good neighbourhood, before going on to build in that new feature.</p> <p class="MsoNormal"><br />However, you can get on top of this and agile techniques can be most effective.<span style=""> </span>For a start – agile methodologies value good code.<span style=""> </span>They also build in checks and balances which look to defend against the failure modes of human beings.<span style=""> </span>This is important.<span style=""> </span>If developers were machine like, they would not care if they were working in a bad neighbourhood and so there would be relatively little impact on the quality of new code introduced there.<span style=""> </span>However humans <span style="font-style: italic; font-weight: bold;">are</span> influenced by their surroundings and your process needs to take this into accounts.<span style=""> </span>A number of XP practices can help here:</p> <ul style="margin-top: 0cm;" type="disc"><li class="MsoNormal" style="">TDD – this is essential to defend against the broken windows initially.<span style=""> </span>You must not add to the problems in your run down area by recklessly adding code.<span style=""> </span>Use the techniques outlined in “Working with Legacy Code” to gradually ‘test infect’ your code base.</li><li class="MsoNormal" style="">Pair programming – two developers have more courage and are more likely to “do the right thing”.<span style=""> </span>Pair programming is a very effective way of cementing the desired values of the team – you can even use this to ‘inject values’ into the team by clever choices of pairs.<span style=""> </span>If you don’t do pair programming – try using design and code reviews to achieve the same effect.</li><li class="MsoNormal" style="">Continuous integration – regular builds maximise the return on investment of the TDD by providing lots of regular feedback.<span style=""> </span>This helps to build momentum which is essential to get is you are to tip your code from a run down neighbourhood into an up and coming district.</li><li class="MsoNormal" style="">Code coverage – not strictly an XP thing – but an important guardian of your defended streets – build this into your CI system.</li><li class="MsoNormal" style="">Acceptance tests – write them for the new features and get them into the build.<span style=""> </span>If you have time you could try to retro fit them but I have never seen this work well.<span style=""> </span>Better to add them as you add new features or modify bugs.</li></ul> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">So – if you have the values and practice in place, how to you ‘tip’ the code base?</p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">If the team values improving the code base and is supported, then you can make tactical improvements.<span style=""> </span>Don’t try to repair the neighbourhood all at once.<span style=""> </span>Rather identify the houses and streets which you repeatedly visit and isolate them.<span style=""> </span>Fix the broken windows by building out unit tests and functional testing to cover just these areas and defend them against the rest of the neighbourhood.<span style=""> </span>Once you have a module under test – defend it with code coverage and automated tests to allow you to spot any breakages and patch them up immediately.<span style=""> </span>Do not allow these clean areas to fall. (todo – relate to the tube trains here).<span style=""> </span></p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal"><b style="">So – we’re all done then…?<o:p></o:p></b></p> <p class="MsoNormal">Well, not really.<span style=""> </span>No neighbourhood stays clean and tidy without ongoing effort.<span style=""> </span>You need to actively guard against the problems which can drag your neighbourhood back down to the skids:</p> <ul style="margin-top: 0cm;" type="disc"><li class="MsoNormal" style="">Maverick coders who churn out vast amounts of poor quality code and are not controlled by their managers or team mates.</li><li class="MsoNormal" style="">Project schedules - prefabs of code are hastily erected to serve a short term need. But are still there 50 years later.<span style=""> </span>This is fine in the short term – but you must plan to repay your design debts in a timely fashion or the surrounding properties in your code will start to suffer.</li><li class="MsoNormal" style="">False values. You will not succeed if your team does not truly believe in the values they are supposed to believe in.<span style=""> </span>This is the biggest killer of all - as people do not perform at their best when their hearts aren’t in the job at hand.</li><li class="MsoNormal" style="">Lack of policing.<span style=""> </span>All too often there is not enough effort put into place to police your new district.<span style=""> </span>If you don’t crack down on the perps who break builds and flout the rest with non-TDD coding, then you will suffer from a rising crime rate, more graffiti and more broken windows - "there goes the neighbourhood...”</li></ul> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal"><br /><o:p> </o:p></p><p class="MsoNormal"><br /><o:p></o:p></p><p class="MsoNormal"><o:p>The inspiration for this entry came from a discussion l</o:p>ast Friday in the pub with Jon. He has already <a href="http://msmvps.com/blogs/jon_skeet/archive/2006/07/14/104665.aspx">blogged about this</a>, but I wasn't going to let that stop me ;-)<br /></p>Stooferhttp://www.blogger.com/profile/03436832701023246750noreply@blogger.com2