Writing a CPU bound script to gauge rough CPU performance

I have wrote a script and running it on different machines. Script looks like below

def f(n):
    x = None
    while n:
        x = simple_math(n)
        n -= 1
    return x

start = now()
f(BIGNUM)    
print now() - start   

At the end of the script it print how much time does it take to finish. Is this good enough to compare different machine for practical CPU speed for simple Python scripts?

By simple I mean it does not use multiprocessing module or any other technique to take advantage of multi-core machines.

This question is not about

• making python programs run faster
• multiprocessing module
• GIL, I/O efficiency etc.
• non cPython programs

Just that I want make sure if my approach to understand CPU performance among 开发者_StackOverflow中文版machines is fairly correct.

What's wrong with all of the countless existing benchmarks? The more sophisticated ones are propably a bit more robust. The major problems of your naive approach I - and I'm not an expert on this topic, mind you - can spot are:

• Modern CPUs are highly complex and employ very clever optimizations. The speed of a purely CPU-bound can vary widely depending on how often the cache can help, how often the program causes pipeline stalls, how often branch prediction is correct, and propably many many more (these were just off the top of my head). Although many of these shouldn't make a difference when you use the same build of the same executable running the same script doing the same pure calculations, they can matter - to a degree none of us can predict - once you change any of these paramteres (e.g. using a different build because of a different OS or architecture).
• Multi-threading OSs will never let a program occupy the CPU exclusively. There will always be some other program running at the same time stealing time, and you can't really know how much of the x seconds were spent running your program and how many were spent on other programs. At the very least, you should run a program many times and take the minimum time as the time it takes with relatively little inference from other programs. And even then, you need to have about the same system load in both benchmarks to make the numbers somewhat meaningful.
• At least CPython won't multi-thread, so you only get the speed of one core.

But since your requirements seem to be "very rough estimate of CPU speed only, in full awareness that these numbers can't be used for anything except putting CPU speed into orders of magnitude, must be taken with a grain of salt even then and don't tell anything about the actual performance of any real applications", it might be okay - just don't consider it anywhere close to accurate. Still, why not use a hardened benchmark suite that already put some effort into mitigating (not removing - nobody can do that) these problems?

Also note that the timeit stdlib module is both easier to use than manually wielding the stopwatch and tries (not too hard, but it's a start) to fix the second point by the method I mentioned.

You can get a rough idea by using these type of methods. But that will not be exact measurement. The execution time of the script will depend on many other things other than CPU speed, like OS and interpreter version used, current system load, memory speed etc. etc. My suggestion is not to depend on this.

EDIT: Just a note. When it comes to performance, many people think only about CPU speed, but actually performance can be hampered by almost everything on the system. For example you have a high speed CPU but low RAM (both in size and speed), then you will get no performance boost up for the CPU.

In essence: No.

Benchmarking is a very difficult problem which usually is not worth solving yourself. It all depends on why you care. Your method will surely give a very rough estimate on if System A is better than System B, but really only when the outcome is vastly different.

What you're trying to do is determine how Real World Application X will perform on different computers. Very rarely is a real world application approximated by a loop of simple math. Even when it is (scientific computing mostly) you're better off measuring times on the actual program.

Real world applications are usually non-linear, and difficult to measure and simulate. Its really one of those problems which has been solved by someone else much better than you could reasonably solve yourself.

If you want a very rough estimate of performance, sure do it your way. Just don't put too much faith in the results because they will be far from what you might call "scientific"

If I understand your intention correctly (and you could clarify it a bit - what is it exactly that you are trying to measure or estimate, processor speed, code speed, something else and for what purpose, but if I understand you then) why not check how is it done in timeit