Can someone suggest a good way to understand how MPI works?

Can s开发者_如何学运维omeone suggest a good way to understand how MPI works?

If you are familiar with threads, then you treat each node as a thread (to an extend)

You send a message (work) to a node and it does some work and then returns you some results.

Similar behaviors between thread & MPI:

They all involve partitioning a work and process it separately.

They all would have overhead when more node/threads involved, MPI overhead is more significant compared to thread, passing messages around nodes would cause significant overhead if work is not carefully partitioned, you might end up with the time passing messages > computational time required to process job.

Difference behaviors:

They have different memory models, each MPI node does not share memory with others and does not know anything about the rest of world unless you send something to it.

Here you can find some learning materials http://www.mcs.anl.gov/research/projects/mpi/

Parallel programming is one of those subjects that is "intrinsically" complex (as opposed to the "accidental" complexity, as noted by Fred Brooks).

I used Parallel Programming in MPI by Peter Pacheco. This book gives a good overview of the basic MPI topics, available API's, and common patterns for parallel program construction.