Existing File based implementation of java.util.Map

I'm working on a project that uses custom Map<String, Entry> (where Entry is a pair of ints) implementation based on B-tree to store from 10 to 100 millions of records, the code for this class is slow and dirty. I need efficient implementation of the Map, which uses a file for storage and a small amount of memory.

I searched and found that Java Edition Of Berkeley DB has java.util.Collection API (including Map), 开发者_开发百科but it seems superfluous to use a fully fledged database for this purpose (it uses directory with many files, has several additional threads for management). Is there a simpler solution?

I had this very same problem recently and looked at everything under the sun, including NoSQL and caches. You want a disk/file based/backed hashmap.

Berkeley DB Java Edition is by far the best. It's fast, scalable, and complete, but you can't distribute it to clients without distributing your source code or buying the commercial version from Oracle.

The only other choice, besides reinventing the wheel, is JDBM2. It also has a hashmap and a tree map. You are responsible for regularly flushing to disk to prevent OutOfMemoryError and it isn't near as fast as Berkeley DB but it is a very good 2nd choice.

Take a look at Kyoto Cabininet, a disk-backed DBM implementation. I've used the previous version, Tokyo Cabinet - it was dead easy to use, basically just like a native Map, and very fast.

JDBM is a lightweight, pure Java B-Tree implementation.