When App Engine (Java) spins up new instances, how do I ensure that Task Queue Memcache crons are shared across them?
This is a problem where my app experiences occasional CPU limit peaks on a site in which almost all of the work is done as a result of a cron task which runs frequentl开发者_StackOverflow社区y every hour to call a task queue which repopulates the memcache.
Recently the site received a large spike in traffic resulting in multiple instances being spun up.
However, to my surprise, I had to reduce the frequency of the Memcache task during those days even though less than 1% of the CPU usage was related to the traffic spike. (99% of the work done in this app is (a) in the memcache task queue and (b) in client-side JavaScript/JSON logic).
What I'm wondering is whether the additional instances that were spun up all require their own Memcache task queues and perhaps this is why I saw a much higher CPU usage related to the task queue that repopulates the memcache during those days - the task was actually being executed separately for each of the instances?
If so, how can I ensure that the task queue - and the Memcache that it populates - is shared across the instances - if this is possible at all?
The task queue and memcache are both global. It's likely that you were enqueueing the task in some fashion that depended on the number of instances, so you were starting more tasks than you should've - it's impossible to say without seeing your code.
精彩评论