How to implement Java/Scala in-memory statistics database?

My need is to aggregate real time statistics of a web application server. For example:

• How many requests of content type X have been done
• How long it takes to process request of type Y

And so on.

This data has to be completely in memory, not in a file, for best performance. It doesn't log each and every request but instead only stores counters of various aspects.

The most easy way I know is to store the开发者_StackOverflow values in a SQL-like table and do SQL-like queries. The benefit is that the indexing is coming off-the-shelf without development effort. I guess some embedded Java databases like Apache Derby would do the work.

The other way to go is to implement collection (say a list) and hash table for each "index column". This way it's all done with Java/Scala collections API, but I actually have to implement indexing mechanism myself, test it, maintain it, etc.

So my question is what way do you think is preferred, and if there are other ways to easily and quickly implement this feature?

Thanks.

I would choose H2 database, I have very positive experiences with it, performance is great as well.

Are you sure that SQL database is well suited for your needs, and have you looked at javamelody, to see if it suits your needs, or if it does not suit you take a look at JRobin for a rolling database implementation.

I would imagine you only need one collection per type of information you need to collection. To improve performance, simplify code I would use TObjectIntHashMap. e.g.

How many requests of content type X have been done

TObjectIntHashMap<ContentType> contentTypeCount 
     = new TObjectIntHashMap<ContentType>();

contentTypeCount.increment(contentType); 

How long it takes to process request of type Y

TObjectLongHashMap<ProcessType> contentTypeTime 
    = new TObjectLongHashMap<ProcessType>();

contentTypeTime.adjustValue(processType, processTime);

I don't see how you can make it any shorter/simpler/faster by using the other approaches you mentioned.

The average time to perform increment(key) on my machines takes 15 ns (billionths of a second)

I also been noticed about Twitter Ostrich that is statistics library for Scala.

It contains counters, gauges and timing meters.

Data is accessible from HTTP REST API.