Document Management System - Database Design

I'm writing my own Document Management System (DMS) in Java (the ones available don't satisfy my needs).

The documents shall be described by the Qualified DublinCore Metadata Standard. The easiest way to do this, in my opinion is do pack the key-value pairs in a RDF model with a XML representation.

To store the metadata for all documents i have two ideas (the document files will be stored in the filesystem):

1. Store all metadata of all documents in a single XML file
2. Make a XML file for each document and store it either in the filesystem or in a RDBMS (like the H2 database engine for Java), a key-value database won't solve this because the keys for one document are not unique.

Since (many) documents are linked among each other the first approach may would be better for analysing the data, but the second approach may be much faster.

Which solution you would recommend? Or are there any better solutions?

Stefan

I don't know how your analysis work, but if you need the complete graph in memory to do your analysis then use variante 1 (Store all metadata of all documents in a single XML file), because you will get no gain (but only extra work) from variante 2 in this scenario.

added

If this extra work for variant 2 is not to much, then I recomend variant 2, because it can be more calable.

• you could update or add document meta data by writing only a small xml file instead of a huge one
• it depends on what xml parser you use, but in some cases it is faster to parse some smaller xml files than one huge one (but this strongly depends on the ammout of data).

Have you considered using MongoDB and GridFS? http://www.mongodb.org/display/DOCS/GridFS+Specification

You can store your documents directly in MongoDB as binary and even store the associated metadata for that particular file in any format you want. It would have the ability to store documents even if they have the same name and it will generate it's own unique IDs.

BTW: even if it does not belong to your question: have a look at a JCR (Java Content Repository) implementation like JackRabbit. You could use it to store your documents and maybe your meta data too.

I'd look into a NO SQL document solution like Couch DB to see if it could help you.

I don't like the file system solution; there's no abstraction whatsoever to help you there.

If your are always accessing all documents, none of your approaches would be slower than the other. But I would recommend the second approach. When it comes to analyzing the data, you'll need to read all documents, so there is no difference if they are in different files or in one file...