Google App engine(python) Updating a db.StringListProperty contention/concurrency issues
Ive been looking at the principles of fan out of messages as described in the google IO "building scalable complex apps"
In it it suggests that using a list property for say a list of receivers is a scalable solution.
In this scenario how does one update the list property so that contention issues don't step in, If the app is handling many users
using the IO example:
class message(db.model)
sender=db.stringproperty()
body=db.textproperty()
class messageindex(db.model)
receivers=db.stringlistproperty()
To adapt their example i would also need
class followers(db.model)
user=db.userproperty()
followers=db.stringlistproperty()
(code is just in for an example and is not typed correctly - sorry)
The concept is if someone follows you, you add their key to your followers list in the followers model, If you make a message, you store your followers list in the message list - and using a simple query all users get the message - pretty simple stuff.
The issue is updating someones list of followers. Assuming that some accounts could have millions of followers - if i simply update the entity their is going to be contention issues - One would also need more than one entry as i think their is a limit of like 5000 entries per list. And of course requests may be sent to "add" or "remove" a person. What would be the best way to do this. I was thinking about using the task_q开发者_StackOverflow中文版ueue service. I was thinking about a work model that stores each follow request and triggers a task to run in say 60 seconds. The task gets all the work to be done for a persons followers list - and builds the new list. Not sure how this would work - but it would stop contention issues as only one thread could execute in one min.
Does anyone have any code examples good advice,help on how i can do this in a scalable manner - i don't think m cache can be used in the method as any loss would mean a follow request could be lost.
I have now found the solution for this using a fork-join-queue. Their is a post on google IO 2010 - regarding how this is done:
link text
精彩评论