Automating Integration Tests: Use xUnit?

I'm looking into how best to automate integration tests (by which I mean complete use cases entirely within our application)

The questions

• Correct Approach for Unit Testing Complex Interactions
• What are the pros and cons of automated Unit Tests vs automated Integration tests?
开发者_如何学编程

cover the "why" and "what" aspects very well.

The question Automated integration testing a C++ app with a database implies that xUnit frameworks are a good way to create and execute integration tests. Are xUnit's really well suited to that task? Are there common gotcha's to be aware of? A good approach to follow?

Are there better approaches (short of possibly purchasing the HP / former Mercury tool suite)?

My specific environment for this project is Java / SpringSource / Hibernate but am also interested in suggestions for the .Net platform.

The question Automated integration testing a C++ app with a database implies that xUnit frameworks are a good way to create and execute integration tests. Are xUnit's really well suited to that task? Are there common gotcha's to be aware of? A good approach to follow?

JUnit and TestNG are initially unit testing frameworks but can be used for integration testing as well. So to me, the answer is yes, they are well suited for integration testing, e.g. testing the service --> domain > persistence --> database layers (I'll actually come back on integration testing later). One of the tricky things when doing integration tests that involve the database is the data. A tool such as DbUnit can help and is typically used to put the database in a known state before to run each test (and to perform asserts on the database content). Another approach is to run the tests in a transaction and to rollback the transaction at the end of the test. Spring allows to do that very easily, so does the unitils library. In any case, a best practice is to avoid interdependent tests as much as possible (as mentioned in the link you gave), they are just a nightmare.

Are there better approaches (short of possibly purchasing the HP / former Mercury tool suite)?

To my knowledge, such tools are more end-to-end testing tools i.e. functional testing tools. So if by integration tests (which for me mean testing several components together) you actually mean functional tests (this is my understanding of complete use cases), I'd suggest to look at things like:

• Abbot, Marathon, Frankenstein, etc for Swing applications
• iMacros, Selenium, Cucumber, etc for Web applications
• SoapUI for Web Services

Pay a special attention to the one in bold (all are actually great tools but I'm sure the one in bold provide good automation capabilities). And because HTTP and SOAP are standards (this doesn't apply to the Swing UI testing tools of course), these tools are not really Java specific (even if the tests themselves are written in Java/Groovy for SoapUI). And BTW, Selenium supports many programming languages).

Multi-threading can be a problem, since JUnit won't pick up exceptions in other threads. There are some Java Puzzlers about that. You also need to invent your own ways of doing statistical testing and the assert methods can be a bit rough. I also think that the semantics of JUnit are a bit unclear (JUnit uses one separate instance per test method for instance). For these reasons I switched to TestNG, which in my opinion is a better designed framework. The fact that JUnit was designed using extreme programming shows sometimes.

As mentioned you can do it with xUnit frameworks, but if you will want to mix Java and .Net, or web applications and desktop applications or add some more complexity to overall picture, than you won't be able to do it with just one unit test framework. So you will need to have many test tools, many test environments, many test script developers (for example one foe Java unit tests, one for .Net tests)... and this will add up complexity, troubles, costs...

As for HP Quick Test Pro you mentioned it should cover most of your needs. I mean most, because there may be some areas where it is not suitable (no way to run scripts on applications via Citrix virtualization), but for the most cases it will do the job. It is suitable for java/.net/web and other things (there are plugins for specialized uses). QTP usually operates on GUI objects so you can prepare test cases for user use cases, and test can be performed in a manner that normal user would perform actions (just a bit faster you have to intentionally slow it down to user speed if needed).
You will probably you will need one tool, one test environment, one test scrip developer (VB). It is expensive, but if it is for company it should be better choice in the long run.
And if you ask from company perspective it will play well with HP Quality Center if you decide to use it for your whole Testing division/team. Unless you use IBM solutions, than they have their own tool suite as part of their software Delivery Platform including Rational Robot