Which is a better sharding strategey with respect to performance? Hashing a GUID or contacting a scale-out manager?
We're building a relatively high profile site that is expected to have 100 million hits on the first days of launch. My predecessor had argued for a scale-up SQL strategy using a single server with 1 TB RAM and 32 cores. We have been advised that this is not a feasible soluiton.
开发者_开发知识库In response, I have shifted to a scale-out strategy with multiple SQL servers and horizontal partitioning. My question revolves around how I will direct the DAL to the appropriate database server. There will be many reads and writes for each user of the application. My first thought was to use a single scale out server that stored the profile id (GUID) of each user and would return the connection string to that user's shard. This seemed like a lot of overhead and created a single point of failure.
My second strategy was to route to the database by GUID so I could directly code this into the DAL. GUID's aren't random though so I'm thinking I'd need to hash it in order to get a relatively even distribution between my database shards. Every user including anonymous users has a GUID, so this is really the only property I have available to me that I can use for sharding.
So the question is whether I'm going to kill performance with the hashing that will have to occur. I'm pretty confident that the hash will be less of a bottleneck than a database read, but I'd really like some feedback on this or any other thoughts the community would like to share about my strategy.
Some specifics: We're using SQL 2008 R2 Enterprise on the db servers. Each db will have 64GB RAM and 8 cores. The databases will be on shared storage. Vmotion will be used if a server goes down. There will be a slew of web servers at launch (30-40?) but the exact number will be dictated by performance testing. The application is built on .net 4.0 with the Enterprise Library v5. Web server load balancing will be handled by a Cisco ACE. We have requested that each of the database servers be on a separate vsphere instance.
Thanks!
Is any user profile needing to interact with another user profile? Typical example would be a Wall of some sort, eg. where you see the status updates from all your friends. See Scale out SQL Server by using Reliable Messaging to understand why I'm asking this.
As for hashing, a good hashing scheme can accommodate a change in the distribution (eg. a segment owned by machine A is split into two new segments now owned by A and B) w/o changes to the application. Hashing in the app layer does not solve this problem, having a dedicated 'scale-out manager' is better, as long as you design the scale-out manager to be highly available and control the number of requests it has to respond to (eg. less thatn 1 per HTTP request on the front end layer).
精彩评论