What do the Appstats values mean?

Take these stats from a post from the App Engine blog as an example:

real = 107ms  
cpu = 141ms  
api = 388ms  
overhead = 1ms

RPC Total: 63ms (388ms api)   
Grand Total: 107ms (530ms cpu + api)

I think I understand overhead: it gives the amount of time taken to write the logs, excluding the time it took to store the logs in memcache.

I am confused by the other numbers:

• What exactly do real, cpu and api mean?开发者_StackOverflow
• How is api different from RPC total?
• What is the "Grand Total"?

This is my understanding:

• real is the time as measured by a clock. This is time elapsed.

• api usage is the time spent on RPC's, such as accessing the datastore. This is not truly a time, but some amount of computing resources measured in time.

• cpu usage is the time spent executing code. Again, it's not really a time but resource usage as measured in time.

• api is different than RPC Total only in that RPC total shows the amount of clock time that has elapsed during the api time. It's possible to do 388ms of computation in 63ms because of parallelism. So, RPC Total shows both clock-time spent, as well as resoure usage.

• Grand Total is the total wall time (the same as real), with the sum of cpu, api, and overhead. In this case, 530ms of quota are used in 107ms.

• overhead is, of course, time "wasted" waiting for "real" work to be done. This mostly includes the resources taken by AppStats itself.

See the document Appstats: RPC Instrumentation for Google App Engine by Guido van Rossum for details.

Guido van Rossum gave a talk at Google I/O 2010 called Appstats - Instrumentation for App Engine where he discusses this briefly. It's a great talk to learn about App Engine, and optimization and instrumentation in general. It's about an hour long.