Are there any simple solutions for distributing computational work in .NET?

So let's say I have some computers at home.

Let's also 开发者_开发知识库say that I have some algorithm I want to run, that generally takes a lot of time to solve. It can be divided in how many parts I want, so I could run part of it in one machine, part of it in another, etc, and in the end i'd just need to merge the results in one of the computers.

My question is if there is any easy and straightforward way in .NET of making use of several computers to do this kind of computations. If yes, how is it called? I don't mean having to code all the IPC code by myself, something similar to BCL's Tasks but that'd allow me to send "work" to other computers, via an IP or something.

Thanks!

In general distributed computing, there are two models: Message Passing and Shared Memory

It sounds like you're interested in Message Passing, because it typically involves multiple computers which communicate over the network. Shared memory uses the memory system to communicate between processes. From my (limited) experience in distributed computing, almost everyone who uses Message Passing as their model has moved to MPI. MPI is a framework that has been implemented in many languages. There are some pretty strong .NET MPI implementations out there.

This whole post assumes you're doing "distributed computing" in a HPC scenario. In such a scenario, you would have many powerful computers, connected over an inter-connection network, in a server room (or similar). If you're looking for some kind of distributed computing between seperated physical machines (such as folding@home), you'll need to find something else or roll your own solution. Therefore, as Leniel Macaferi mentions in his response, you'll need to ensure your data center is running some version of Microsoft HPC Server. Most HPC clusters run some flavor of linux, which will obviously prevent you from deploying a .NET based solution.

That's a perfect match for Microsoft HPC Server distro as it is tuned specially to address this kind of situation.

I don't know anything that can do it purely based on .NET. That's possible of course, but it would require some coding.

Although not what you want, I'd recommend you read about the .NET Task Parallel Library that helps you divide your workload automatically across the computer's available CPUs (cores) quickly and painlessly.

Interesting articles related to distributed computing:

C# Remoting and Distributed Computing

Legion: Build your own virtual super computer with Silverlight

Windows Communication Foundation (WCF)

Microsoft has in their research labs a Map reduce implementation of Linq called Dryad Linq (currently the beta is only available on HPC servers) but this looks very interesting and does what you are looking for.

Microsoft HPC Server requires a server OS. From the sounds of it, you are not running a server OS. Assuming you are running a desktop version of Windows with .NET 4.0 installed, you could try one of these options:

1. You could manually split up the work and execute the pieces manually.

2. You could use something MPAPI and code up your own nodes and workers.

3. You could write a console application and use DuoVia.MpiVisor to distribute and execute on your workstations. (Full disclosure: I am the author of MpiVisor)

Doing distributed computing in a casual fashion without a lot of fuss is not as easy as one might thing, but either of these two options should get you there.