FoneMonkey test crash during Jenkins run

I use FoneMonkey + Jenkins for automated iOS test. I ask FoneMonkey to generate XML reports (FM_ENABLE_XML_REPORT - set to TRUE) and run it with waxsim. Today I find out that when app crashes (ie due to NSInvalidArgumentException) the xml report is not generated. And then in Jenkis I see 100% passing (I use standard JUNIT plugin), with its obviously not true.

The only ways to spot this is to: a) analyse Jenkins console log (ie looking 开发者_JS百科for *** Terminating app due to uncaught exception) b) spot that in Jenkins report the number of run test is lower then perviously (that's how my teammates spotted the problem).

I know that I can use Jenkins Text Finder plugin for solution a), but I would like to have accurate reports.

Is there any way to achieve it? If no, is FoneMonkey anything useful?

If executing a tool fails (indicated by a non-zero exit code), Jenkins detects this and marks the build as failed. If the tools are broken, there's not much Jenkins can do to help.

For your situation, I see several possibilities:

• FoneMonkey should be fixed to return a non-zero exit code when the app under test crashes
• FoneMonkey should be fixed to always output XML, even when the app under test crashes
• You could write a custom FoneMonkey-output-to-JUnit-parser rather than relying on FoneMonkey (e.g. similar projects exist for parsing Android test runner output)