开发者

Google App engine(python) Updating a db.StringListProperty contention/concurrency issues

Ive been looking at the principles of fan out of messages as described in the google IO "building scalable complex apps"

In it it suggests that using a list property for say a list of receivers is a scalable solution.

In this scenario how does one update the list property so that contention issues don't step in, If the app is handling many users

using the IO example:

class message(db.model)
    sender=db.stringproperty()
    body=db.textproperty()

class messageindex(db.model)
    receivers=db.stringlistproperty()

To adapt their example i would also need

class followers(db.model)
    user=db.userproperty()
    followers=db.stringlistproperty()

(code is just in for an example and is not typed correctly - sorry)

The concept is if someone follows you, you add their key to your followers list in the followers model, If you make a message, you store your followers list in the message list - and using a simple query all users get the message - pretty simple stuff.

The issue is updating someones list of followers. Assuming that some accounts could have millions of followers - if i simply update the entity their is going to be contention issues - One would also need more than one entry as i think their is a limit of like 5000 entries per list. And of course requests may be sent to "add" or "remove" a person. What would be the best way to do this. I was thinking about using the task_q开发者_StackOverflow中文版ueue service. I was thinking about a work model that stores each follow request and triggers a task to run in say 60 seconds. The task gets all the work to be done for a persons followers list - and builds the new list. Not sure how this would work - but it would stop contention issues as only one thread could execute in one min.

Does anyone have any code examples good advice,help on how i can do this in a scalable manner - i don't think m cache can be used in the method as any loss would mean a follow request could be lost.


I have now found the solution for this using a fork-join-queue. Their is a post on google IO 2010 - regarding how this is done:

link text

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜