Do testing frameworks exist that allow a percentage of failure?

When reading a question about testing a Java program several times with random seeds the word testing sparked of the association with unit testing in my mind, but that might not have been the kind of testing going on there.

In my opinion introducing randomness in a unit test would be considered bad practice, but then I started considering the case where a (small) percentage of failure might be acceptable for the moment.

For example, the code fails to pass the unit test once every 10^n for n > 3 and gradually you want n to go to 开发者_StackOverflow中文版infinity without the test going red, maybe yellowish.

Another example might be a system-wide test where most of the time things go right, but you still want to limit/know how often they might go wrong.

So my question is, are there any frameworks (full spectrum of testing) out there that can be persuaded to allow a percentage of failure in an huge/excessive amount of repeated tests?

You can "persuade" most testing frameworks to allow partial failures by performing your "tests" not using the framework directly but just by doing plain old conditional (eg: if-statements) and recording the percentage of failures. Then use the framework to assert that this percentage is below your threshold.

If your tests are deterministic (which is generally considered to be a good thing) then another approach is to test for the current behavior even when it's wrong, but to comment the incorrect assertions (typically with what the "right" answer should be). If these tests ever fail you can check the comment. If the code has become "correct" then great, update that assertion. If not, then decide whether the change in behavior is better or worse than the old behavior, and act accordingly. This approach lets you sort of "tighten the screws" over time.