Looking for "test execution manager" software to manage automated tests [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.

Closed 8 years ago.

Improve this question

We develop several products and already have extensive unit-tests and fully automated functiona开发者_StackOverflowl tests for them. Problem is that those tests don't run frequently, just manually by developer or just before shipping a new version.

I'm looking for a "test execution manager" software which will allow:

• defining test suites as a collection of my existing tests ;
• executing the test suites on multiple machines in a our test lab ;
• collecting results and presenting them nicely ;
• preserve test execution history and results

Most "testing solutions" I've found concentrate on "writing automated tests" (which we already have working) or closely integrate with other aspects of software development, like defining requirements and filing bugs (which we have and don't want to change).

• Can anyone recommend a simple and flexible software to do the above without forcing specific development processes?
• I though on using (or abusing) Hudson CI for this. Hudson can already run tests, collect results and present them, both periodically or due to code commit; but it was not designed for test suite definition. Any input from experienced Hudson users on this idea is appreciated..

---

First of all, our developers are not allowed to check in code without running the unit tests. We also run a CI server (Hudson), which builds after a commit and runs the unit tests. We are working on getting the functional tests implemented for the nightly builds.

You said your developers test the software? This is a bad thing. At least let a developer that is not familiar with the code to test your app otherwise you are likely to overlook some bugs, because their existence was ruled out by the developer writing the code. Additionally, who writes the functional tests? Developers again? You should get your BA's to write them. Always remember, four eyes see more than two.

So after all that said, I assume, that the unit tests, will always be run before code is checked into your SCM. The following is targeted primarily at the functional tests.

Simple solution:

• You can always create scripts to bundle your tests (batch or shell script that runs the individual test).
• Executing of test suites is actually one of the purposes of Hudson
• Collecting and presenting results, that is what Hudson is for
• See above, can be done with Hudson, without abusing it.

A good solution:

Did you look at tools like IBM Rational Quality Manager? Depending on the test tools you use, you might want to use a test management tool different one. Oracle also offers a tool for it. Don't be mistaken usually these tools can be fairly expensive and offer way more than you want to use. With a little bit help from google you should find something that suits your needs. My keywords were "centralized test management".

In case you use FitNesse for your functional test. You can define suites in FitNesse and I think a suite can be part of a larger suite. FitNesse definitely keeps historic test data. The test can be run from command line which enables you to run the tests from ant or maven.

If you use a unit test framework for your functional testing you can also run them as part of an nightly build and schedule it using your CI server (Hudson or Cruise Control or ...)