Running a large IN query (searching for users with an ID list) on GAE

I am trying to detect, after a user registers, which friends from Facebook have already registered for my service. My current implementation is very CPU intensive:

for eachFriend i开发者_C百科n facebookFriends:
  friendUser = User.get_by_key_name(eachFriend['id'])
  if friendUser:
    friendUsers.append(friendUser)

I tried optimizing the query by using the IN operator:

 users = User.all().filter('id IN', idList).fetch(10) # the idList is the list of IDs for a users facebook friends

This method fails as the maximum subqueries of the IN operator is 30.

Any tips?

Using an IN operator actually makes your query less efficient: Instead of doing a fast get operation for each friend, you're doing a slow query operation (IN and != filters are broken out into multiple queries on the backend).

Instead, do a single batch fetch for all the matching suers:

friendUsers = User.get_by_key_name([x['id'] for x in facebookFriends])

This returns a list of all friends, with None values for any friends that don't yet exist.

Yeah, you can have each registered user store his friends in a ListProperty so that when I register, you can do an = query on that property to see who has me as a friend. = queries on ListProperties return all entities having the filtered-on value anywhere in the list, and they don't generate subqueries the way IN queries do.

Just be aware of per-entity index limits if some of your users have tons of friends.