Concurrent programming techniques, pros, cons

There is at least three well-known approaches for creating concurrent applications:

1. Multithreading and memory synchronization through locking(.NET, Java). Software Transactional Memory (link text) is another approach to synchronization.

2. Asynchronous message passing (Erlang).

I would like to learn if there are other approaches and discuss various pros and cons of these approaches applied to large distributed applications. My main focus is on simplifying life of the programmer.

For example, in my opinion, using multiple threads is easy when there is no dependencies between them, which is pretty rare. In all other cases thread synchronization开发者_运维问答 code becomes quite cumbersome and hard to debug and reason about.

I'd strongly recommend looking at this presentation by Rich Hickey. It describes an approach to building high performance, concurrent applications which I would argue is distinct from lock-based or message-passing designs.

Basically it emphasises:

• Lock free, multi-threaded concurrent applications
• Immutable persistent data structures
• Changes in state handled by Software Transactional Memory

And talks about how these principles influenced the design of the Clojure language.

Read Herb Sutter's Effective Concurrency column, and you too will be enlightened.

With the Java 5 concurrency API, doing concurrent programming in Java doesn't have to be cumbersome and difficult as long as you take advantage of the high-level utilities and use them correctly. I found the book, Java Concurrency in Practice by Brian Goetz, to be an excellent read about this subject. At my last job, I used the techniques from this book to make some image processing algorithms scale to multiple CPUs and to pipeline CPU and disk bound tasks. I found it to be a great experience and we got excellent results.

Or if you are using C++ you could try OpenMP, which uses #pragma directives to make loops parallel, although I've never used it myself.

In Erlang and OTP in Action, the authors present four process communication paradigms:

• Shared memory with locks

  A construct (lock) is used to restrict access to shared resources. Hardware support is often required from the memory system, in terms of special instructions. Among the possible drawbacks of this approach: overhead, points of contention in the memory system, debug difficulty, especially with huge number of processes.

• Software Transactional Memory

  Memory is treated as a database, where transactions decide what to write and when. The main problem here is represented by the possible contentions and by the number of failed transaction attempts.

• Futures, promises and similar

  The basic idea is that a future is a result of a computation that has been outsourced to a different process (potentially on a different CPU or machine) and that can be passed around like any other object. In case of network failures problem can arise.

• Message passing

  Synchronous or asynchronous, in Erlang style.