Pro's and Con's of unit testing after the fact

I have a largish complex app around 27k lines. Its essentially a rule drive multithreaded processing engine, without giving too much away Its been partially tested as it's been built, certain components.

Question I have, is what is the pro's and con's of doing unit testing on after the fact, so to speak, after its been implemented. It is clear that traditional testing is going to take 2-3+ months to test every facet, and it all ne开发者_开发知识库eds to work, and that time is not available really.

I've done a fair bit of unit testing in the past, but generally it's been on desktop automation or LOB apps, which are fairly simple. The app is itself is highly componentized internally, interface driven really. I've not decided on what particular framework to use. Any advice would be appreciated.

What say you.

I think there are several advantages to unit testing existing code

• Regression management
• Better understanding of the code. Testing it will reveal cases you did not anticipate and will help define the behavior of the code
• It will point out design deficiencies in the code as you stuggle to test poorly defined methods.

But I think it's more interesting to consider the cons of unit testing code. AFAIK, there are no cons. All of the time spent adding tests will pay for themselves even in everything but the shortest of time cycles.

There are many reasons to unit test code. The main reason I would advocate unit testing after the fact is simple. Your code is broken, you just don't know it yet.

There is a very simple rule in software. If the code is not tested, it's broken. This may not be immediately obvious at first, but as you begin testing, you will find bugs. It's up to you to determine how much you care about finding these bugs.

Besides this, there are several other important benefits of unit testing,

• regression testing will be made simpler
• other developers, that are less knowledgeable, can't break your desired behavior
• the tests are a form of self documentation
• can reduce time in future modifications (no more manual testing?, less bugs?)

The list can go on and on. The only real drawback is the time it takes to write these tests. I believe that drawback will always be offset by the time it takes you to debug problems you could have found while unit testing!

Depending on how many bugs "manual testing" turns up, you could simply do test-driven bug fixing which in my experience is far more effective than simply driving up code coverage by writing "post-mortem" unit tests.

(Which is not to say writing unit tests afterwards is a bad idea, it's just that TDD is almost always a better idea.)

Here's a few of each to my mind:

Pro:

• Time is saved in not having to test methods that have been removed as the design evolved over time. What is left is what really has to get tested.
• By adding tests, this allows an opportunity to review all the aspects in the app and determine what other optimizations one could add now that a working prototype is ready.

Con:

• Large time investment to get the tests written, new functionality may be delayed for some time to generate all the tests.
• Bugs may have been introduced that the tests will discover that may cause this to be longer than initially planned.

The main point would be that adding unit tests allows for refactoring and putting more polish on the application.

I think one of the biggest con of testing "after the fact" is that you will probably have a harder time testing. If you write code without tests, you usually don't have testability in mind and end up writing code that is hard to test.

But, after you spent this extra time writing tests and changing your code for better testability, you'll be much more confident about making changes, once you won't need a lot of time to debug and check if everything is ok.

Finally, you might find new bugs which weren't caught before, and spend some time fixing it. But hey, that's what tests are for =)

Pro post facto unit testing:

• Get documentation you can trust.
• Improve understanding of the code.
• Push toward refactoring and improving the code itself.
• Fix bugs that lurk in the code.

Con post facto unit testing:

• Waste time fixing bugs you can live with. (If you wrote 27KLOC, we hope it does something, right?)
• Spend time understanding and refactoring code you don't need to understand.
• Lose time that could go into the next project.

The unasked question is just how important an asset is this code to your organization, long term? The answer to this question determines how much you should invest. I have plenty of (successful) competitors where the major purpose of their code is to get out numbers to evaluate some new technique or idea. Once they have the numbers, the code is of little marginal value. They (rightly) test very carefully to make sure the numbers are meaningful. After that, if there are fifty open bugs that don't affect the numbers, they don't care. And why should they? The code has served its purpose.

If you are doing any refactoring, those tests will help you detect any bugs that will appear in the process.

Unit testing "after the fact" is still valuable, and provides most of the same advantages of unit testing during development.

That being said, I find it's more work to test after the fact (if you want to get the same level of testing). It's still valuable, and still worth while.

Personally, when trying to tackle something with limited time, I try to focus my testing efforts as much as possible. Any time you fix a bug, add tests to help prevent it in the future. Any time you're going to refactor, try to put enough testing in place to feel confident you're not going to break something.

The only con of adding unit testing is that it does take some development time. Personally, I find that the development time spent on testing is far outweighed by the time saved in maintenance, but this is something you need determine on your own.

Unit testing is still definitely useful. Check out http://en.wikipedia.org/wiki/Unit_testing for a full list and explanation of the benefits.

The main benefits you will gain are documentation, making change easier, and it simplifies future integration.

There are really no costs to adding unit testing except your time. Realize though that the time you spend adding unit testing will reduce the amount of time you will need to spend in other areas of development by at least the same amount and most likely more.

Unit testing doesn't prove that a system works. It proves that each unit works as an independent unit. It doesn't prove that the integrated system will work

Unit testing "after the fact" is useful for two things - finding bugs that you've missed so far and won't find using any other kind of testing (especially for rare conditions - there's huge numbers of rare conditions that can happen in particular units for any real world system), and as regression tests during maintenance.

Neither of these is going to help much in your situation - you need to do other forms of testing either way. If you don't have time to do what you need to do, taking on even more work is unlikely to help.

That said, without unit testing, I guarantee you will have nasty surprises when the customers start using the code. It's all those rare conditions - there's so many of them that some of them are bound to occur soon. Black-box testers tend to get into habitual patterns, which mean they only test so many rare cases - and they have no way of knowing what rare cases there are in particular units and how to trigger them anyway. More users means more variations in usage patterns.

I'm with those who say unit tests should be written as part of the programming process - one of the programmers responsibilities. As a rule, code gets written faster that way, as you get fewer and less complex bugs to track down as you go, and you tend to find out about them when you're still familiar with the code that has the bug.

If development is "done" I would say that there is not too much point in unit testing.

This is one of these difficult value judgement types of questions.

I would mostly agree with Epaga, that writing new tests as you fix bugs (perhaps with a couple of extra tests thrown in) is a good approach.

I would add two further comments:

• Doing backed-off black box testing to a unit before making large changes can be a good idea
• Consistency testing isn't unit testing, but certain types of program lend themselves to the easy generation of consistency tests. This might be one approach to making sure you don't break things.