Caliper: How to run multiple benchmarks?

I have written a few benchmarks using Caliper. How can I run multiple benchmarks at once? I currently have many classes that extend SimpleBenchmark and have a few timeXXX methods each. Is it possible to run these all at the same time?

SimpleBenchmark's javadocs say:

The benchmarks of a suite are defined by . They may be static. They are not permitted to take parameters . . ..

Uh, what? This looks incomplete.

Caliper's mailing list says it is not for Q&A, and to post a question here with the tag "caliper". However, I see zero SO posts with 开发者_运维知识库this tag! Since I don't have 1500 rep yet, I can't create a new tag.

Use case: There is only one setUp/tearDown method. I have many tests in my benchmark class, each of which allocates a lot of memory. I currently have to use a gargantuan heap, which only gets bigger as I add more tests. Also, many tests in one file becomes somewhat untidy to maintain. It would be nice to specify multiple benchmark classes and have all tests in all of them be considered as one run.

No. There's no built-in mechanism to run unrelated benchmarks as a suite. You can accomplish this with a simple shell script:

#!/bin/bash
echo $* | xargs -n 1 $CALIPER_HOME/caliper

What's your use case? We'd like to have better support for running benchmarks in Hudson etc; hopefully we can put that together soon.