SQLite Abnormal Memory Usage

We are trying to Integrate SQLite in our Application and are trying to populate as a Cache. We are planning to use it as a In Memory Database. Using it for the first time. Our Appli开发者_StackOverflowcation is C++ based.

Our Application interacts with the Master Database to fetch data and performs numerous operations. These Operations are generally concerned with one Table which is quite huge in size. We replicated this Table in SQLite and following are the observations:

Number of Fields: 60 Number of Records: 1,00,000

As the data population starts, the memory of the Application, shoots up drastically to ~1.4 GB from 120MB. At this time our application is in idle state and not doing any major operations. But normally, once the Operations start, the Memory Utilization shoots up. Now with SQLite as in Memory DB and this high memory usage, we don’t think we will be able to support these many records.

Q. Is there a way to find the size of the database when it is in memory?

When I create the DB on Disk, the DB size sums to ~40MB. But still the Memory Usage of the Application remains very high. Q. Is there a reason for this high usage. All buffers have been cleared and as said before the DB is not in memory?

Any help would be deeply appreciated.

Thanks and Regards Sachin

A few questions come to mind...

What is the size of each record?

Do you have memory leak detection tools for your platform?

I used SQLite in a few resource constrained environments in a way similar to how you're using it and after fixing bugs it was small, stable and fast. IIRC it was unclear when to clean up certain things used by the SQLite API and when we used tools to find the memory leaks it was fairly easy to see where the problem was.

See this:

PRAGMA shrink_memory

This pragma causes the database connection on which it is invoked to free up as much memory as it can, by calling sqlite3_db_release_memory().