Reliable and efficient way to handle Azure Table Batch updates

I have an IEnumerable that I'd like to add to Azure Table in the most efficient way possible. Since every batch write has to be directed to the same PartitionKey, with a limit of 100 rows per write...

Does anyone want to take a crack at implementing this the "right" way as referenced in the TODO section? I'm not sure why MSFT didn't finish the task here...

Also I'm not sure if error handling will complicate this, or the correct way to implement it. Here is the code from the Microsoft Patterns and Practices team for Windows Azure "Tailspin Toys" demo

    public void Add(IEnumerable<T> objs)
    {
        // todo: Optimize: The Add method that takes an IEnumerable parameter should check the number of items in the batch and the size of the payload before calling the SaveChanges method with the SaveChangesOptions.Batch option. For more information about batches and Windows Azure table storage, see the section, "Transactions in aExpense," in Chapter 5, "Phase 2: Automating Deployment and Using Windows Azure Storage," of the book, Windows Azure Architecture Guide, Part 1: Moving Applications to the Cloud, available at http://msdn.microsoft.com/en-us/library/ff728592.aspx.

        TableServiceContext context = this.CreateContext();

        foreach (var obj in objs)
        {
            context.AddObject(this.tableName, obj);
        }

        var saveChangesOptions = SaveChangesOptions.None;
        if (objs.Distinct(new Partit开发者_JS百科ionKeyComparer()).Count() == 1)
        {
            saveChangesOptions = SaveChangesOptions.Batch;
        }

        context.SaveChanges(saveChangesOptions);
    }


   private class PartitionKeyComparer : IEqualityComparer<TableServiceEntity>
    {
        public bool Equals(TableServiceEntity x, TableServiceEntity y)
        {
            return string.Compare(x.PartitionKey, y.PartitionKey, true, System.Globalization.CultureInfo.InvariantCulture) == 0;
        }

        public int GetHashCode(TableServiceEntity obj)
        {
            return obj.PartitionKey.GetHashCode();
        }
    }

Well, we (the patterns & practices team) just optimized for showing other things we considered useful. The code above is not really a "general purpose library", but rather a specific method for the sample that uses it.

At that moment we thought that adding that extra error handling would not add much, and we diceided to keep it simple, but....we might have been wrong.

Anyway, if you follow the link in the //TODO:, you will find another section of a previous guide we wrote that talks a little bit more on error handling in "complex" storage transactions (not in the "ACID" form though as transactions "ala DTC" are not supported in Windows Azure Storage).

Link is this: http://msdn.microsoft.com/en-us/library/ff803365.aspx

The limitations are listed in more detail there:

• Only one instance of the entity should be present in the batch
• Max 100 entities or 4 MB payload
• Same PartitionKey (which is being handled in the code: notice that "batch" is only specified if there's a single Partition key)
• etc.

Adding some extra error handling should not overcomplicate things too much, but depends on the type of app you are building on top of this and your preference to handle this higher or lower in your app stack. In our example, the app would never expect > 100 entities anyway, so it would simply bubble the exception up if that situation happens (because it should be truly exceptional). Same with the total size. The use cases implemented in the app make it impossible to have the same entity in the same collection, so again, that should never happen (and if it happens, it wouls simply throw)

All "entity group transactions" limitations are documented here: http://msdn.microsoft.com/en-us/library/dd894038.aspx

Let us know how it goes! I'm also interested to know if other pieces of the guide were useful for you.