Design scalable highly updated ressource on a DHT in key/value pair
I would like to design a resource that users would continuously update and read, this resource need not to be always update but must scale well, I mean that the nodes responsible for the resource and his replicas should not be overloaded.
The main problem is that I cannot see how it is possible! I could offload these nodes开发者_开发技巧 by adding read cache and update them at a slow pace, but for the writing I have no idea how to scale because values must be at known keys to be recovered by the users and so I can't share the load on the DHT...
Thx a lot for your ideas!
After some though,
one possible idea would be to make something approximative, each time someone want to post a feed it also post the feed with a key equal to a timestamp representing the second (like 25/03/2011 at 5 min and 1 sec) at which he posted the feed, so that the load is automatically shared between the node and that the responsible node quickly change.
When someone want to know the last feeds posted he just check the timestamp of the actual second and the 2-3 past second and can thus recover the last feeds distributing the load between the nodes responsible for the different seconds...
That should make the job I think :).
Hope my own answer will help someone later ;)
精彩评论