Synchronize Berkeley DB with Mysql

I have a legacy system that has been ported to a modern system and uses berkeley database. It is compiled with open cobol.

In the previous system the serialized fixed data CISAM files were converted one time per day to microsoft database using ACCESS macros in order to be used from a simple web front end.(one way) I want to create a new web front end and i want data to be real time updated...

The problem is that berkeley database is key value database and doesnt support sql.

So the problem here is Should i create a web service that provides data to the web front end using Berkeley database ? OR Should i synchronize berkeley database with a relational Database (Mysql,postgresql)

In the first case there wouldnt be a synchronization issue.But the problem is that flexibility is very limited... For example sorting by a collumn that it is not indexed requires custom sorting function... Summing some fields requires reading all the record data and then summing manually..

In the second case the simple way is to just read and convert to database . I cant figure out a way to check what has been updated...inserted or deleted.

But i tend more to second method.. using the following algorithm Check every 5 minutes the file modifation date... Open files that have been modifie开发者_Python百科d... Then get all the keys and compare to cached keys array. Then loop through each record and check if data hash is the same with cached hash.

Then delete/insert ..

ANy other ideas?

Why not take the source code to the berkely-db database driven system, and find a way to write a SOAP query and handle it using that application. Now you have a web service that can read this BSDDB directly and you don't need synchronization.

Even if you DON'T have the source code for the original application, if you can figure out how to unpack the content of the BSDDB key/value store yourself, you could easily (in Python for example) write a simple web service app that serves up the contents of your BSDDB database using SOAP. I would start by writing a few lines of Python that open up the BSDDB value, and dumps the Key-values, and their associated payloads. (The "Value" values, if you like).

Your choices are to have two repositories and synchronize them or make your web application display and store data directly to/from the Berkeley DB key-value data store.

If your your decision is to maintain and synchronize the data stored in a key-value format within Berkeley DB with a separate RDBMS repository then you'll need to write an application that can perform the synchronization for you. The mapping between the key-value data store and the RDBMS data store will need to be performed by your synchronization application along the lines that you've already outlined. You can either scan/compare/contrast the two repositories as you propose or create an "application data log" that can be read and applied by your synchronization. The problem with this approach is that you may end up spending a lot of time and effort trying to synchronize the two repositories.

Another option is to write your Web application so that it can communicate directly with the Berkeley DB respository (as suggested by Warren). There are several scripting languages that have Berkeley DB bindings (Perl, PHP, Python, TCL and Ruby to mention a few). If having the web application communicate with the Berkeley DB repository directly is a problem, you can always write a simple server application which talks to the BDB repository and that can process HTTP requests from the web application (basically, the SOAP solution suggested by Warren).

I think that you'll find that having a single repository is much easier to manage in the long run, even if it means some additional application development work in the short run.

Well, Berkeley DB does have an SQL API (since spring 2010). Please see http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html for more details.

Also, I'd recommend posting your question to the Berkeley DB forum http://forums.oracle.com/forums/forum.jspa?forumID=271