C embedded automatic unit test generation

Is there any SW to generate unit tests in C and embedded applications? The reason I am asking is that my boss told me he heard from someone that "You need a tool to analyze the code and create 80% of all relevant testcases automatically, the remaining 20% you use all your time and focus on", else it would take "too much time".

I am very skeptic about this statement and can't see clearly what kind of tests that could be auto generated and if they would be any good at all.

I can, however, see that it would be possible to generate interface unit tests automatically for the API:s.

So can someone enligh开发者_高级运维ten me on this issue?

I recommend API Sanity Checker tool:

An automatic generator of basic unit tests for a shared C/C++ library. It is able to generate reasonable (in most, but unfortunately not all, cases) input data for parameters and compose simple ("sanity" or "shallow"-quality) test cases for every function in the API through the analysis of declarations in header files.

The quality of generated tests allows to check absence of critical errors in simple use cases. The tool is able to build and execute generated tests and detect crashes (segfaults), aborts, all kinds of emitted signals, non-zero program return code and program hanging.

Unique features:

• Automatic generation of input arguments and test data (even for complex data types)
• Modern specialized types instead of fixtures and templates

See examples for FreeType2.

I'm author of this project and you can ask me any questions on it.

Your boss has got the wrong end of the stick.

I know of no tools that will generate unit tests for you.

What he may be mistaking is code coverage and unit testing. While related they are in fact separate issues.

Code coverage will instrument your code and once finished running give you the low down on how much of your source was used in the run. This is useful when unit testing as it will effectively show you where you have tested and where you need to focus your work.

It is fairly easy to get the first two thirds of code covered but diminishing returns means to get near a magic 100% takes a lot of time and effort.

Googling "unit test generator" turns up a lot of things, but I do not know if they are any good, or if they'll suit your case.

It is not unit testing, but you can do some code checking with lint or related tools. See: http://www.lysator.liu.se/c/ten-commandments.html I think a current open source tool is splint http://www.splint.org/

Jon Bentley's books have some good discussion of the role of "scaffolding" code, including test scaffolds.

First of all, what do you mean by unit test and generate unit tests?

Do you mean generate a framework, a test harness or do you mean generate a test with data and data checks or assertions that actually calls your code. And, in the latter case, how is that test generated?

More fundamentally, why are you testing? Are you following a standard that requires a certain level of testing, or are you 'just' trying to reduce risk and cost in later stages of development? Or maybe you are building on an existing system and just want to make sure you do not break any existing functionality.

A previous answer mentioned Cantata, we have recently released a new version with a component called 'baseline testing'. This may be just what you are looking for. It will create, for C code, a set of unit tests, containing test cases that stand a good chance of fully exercising each of your source files. The tool achieves this by reading your source and producing a set of test that drive the execution down each path aiming to achieve your desired coverage target – 100% statement, decision or even MC/DC coverage. The intent is to 'baseline' your source code as part of the ongoing development of a legacy system, or to fill in coverage gaps after functional or maybe system testing.

See the Cantata++ webpage for more information (and free evaluations)

We use CANtata here where I work for generating unit tests / code coverage. Its decent, though I imagine it is a bit pricey.

We use IBM RTRT

http://www-01.ibm.com/software/awdtools/test/realtime/index.html

Although in our use case we don't use it to generate the tests, but I saw some possibilities to generate at least skeleton.

Is there any SW to generate unit tests in C and embedded applications?

Yes. Rational Test Realtime of IBM will be the good choice.

a tool to analyze the code and create 80% of all relevant testcases automatically

No. There is no tool that can do this work for all type of C source codes.

But yes for some cases. For example, in my case, I have big amount of C source codes are required to be tested by customer. But because every source file are quite similar, so we make a small tool that read each file and generate test case (to the scripting language of the Unit Testing Tool) and then execute by the Unit Testing Tool. And yes, in this case, it saves 80% effort.

So you could consider to do this, find out the similar source codes, and make your own tool that can generate test case for the similar.

1. I used Parasoft C++Test. It's quite pricy but gets the job done and fast. Generates all the unit tests and mock functions for both C and C++ and gets you to 100% code coverage. It also comes with its own unit tests framework and mocking framework so that if you want to write your own unit tests you can also do it.
2. Another tool is Tessy which isn't cheap either. It also generates unit tests and mock functions for your entire code. Tessy supports C and C++ but from my brief experience with it, it is more friendly for C code. In cases of C++ you might need to dig into customizations in order to get the generation of tests to work properly.