Looking for data storage... which to choose

I have an application where users can define their own data sets (fields, fields types and such) and then store their data... very similar to them creating and managing their own tables.

Doing this seems to prese开发者_运维技巧nt issues when trying to set it up on something like MySQL... from a custom query perspective and from a storage perspective. I don't want to end up with thousands of tables or even managing so many different databases.

Someone told me that NoSQL was something to look into based on the flexibility of getting away from crazy complex queries.

The end result is the user will be able to query theses datasets to build graphs. Will something like http://redis.io accomplish this task for me?

If not, does anyone have suggestions on the best option to support this task?

Thanks!

You'll need to consider your data model and your desired queries in some detail to make this decision - each of the various NoSQL technologies has a slightly different data model and feature set.

A key-value database such as Cassandra would probably support on-the-fly field definition, but wouldn't support much in the way of field typing. You could store raw byte values and overlay your own type system, but you'd get no support from the database for enforcing types.

NoSQL databases generally don't support complex queries (no joins etc) so you must manage with simple queries (key lookups) or denormalise to support specific queries.

If you are working with graphs, have you considered an RDF database (triple store)? These also allow great flexibility, but are not table-based (relational). They generally support the SPARQL query language. See the http://answers.semanticweb.com/ site.

Based on your description, you need either a document oriented database or a key / value store with an ability to know (possibly even index) values.

Riak would fit that model, as it is a key / value store, where you do not have to predefine structure for those values + it has secondary indicies, where with each {key, value} pair persisted, you can add a custom index. In Riak words you have an ability to: tag a Riak object with some index metadata, and later retrieve the object by querying the index, rather than the object's primary key

which fits the description of what you are looking to solve pretty nicely.

Here is an example from Basho's blog ( simple curl HTTP request ):

curl -X POST \
-H 'x-riak-index-twitter_bin: rustyio' \
-H 'x-riak-index-email_bin: rusty@basho.com' \
-d '...user data...' \
http://localhost:8098/buckets/users/keys/rustyk

which says, insert ...user data... under a key rustyk, or and by the way, tag (read index) it with twitter "rustyio" and email "rusty@basho.com" ( _bin, just means these indicies are binary )

Now to read keys by just created "index", you can simply:

curl localhost:8098/buckets/users/index/twitter_bin/rustyio

which returns:

{"keys":["rustyk"]}

the key that you can use to retrieve that ...user data...