The relationship between bank conflict and coalesced access in CUDA

I try to transfer 开发者_如何学运维some data from shared memory to global memory. Some consecutive threads will access one bank (but not the same 32 bits). So there are some bank conflicts. (I use Visual Profiler to check this) However, those data also be coalesced and then be transfered to global memory. (I use Visual Profiler to check this) Why are the data wrote into global memory with coalesced way? In my opinion, a streaming multi-processor pops the 32-bit word one by one (based on bank's bandwidth). So the memory transactions can not be coalesced in global memory. I may make some mistakes here. Please help to find the mistakes out or give me a reasonable explanation. Thank you.

You have two different things going on here: the read, which incurs a bank conflict, and the write, which may not be coalesced. Since shared memory is much much faster than global, you usually need to worry about the coalesced access first.

Coalescing means that the threads are writing into a small range of memory addresses. For example, if thread 1 writes to address 1 and thread 2 to address 2, this is good. If the write to addresses* 1 and 4, respectively, this is worse. Less importantly, it's optimal if threads write into increasing addresses starting at a multiple of 32, for example addresses 32 and 33. *(I use "address" loosely here, meaning a 4-byte offset).

Bank conflicts occur when multiple threads access shared memory addresses that have the same lower bits (specifically, are equivalent mod 16). If two threads use the same bank, then they will be serialized, meaning one will execute after the other, instead of both accessing memory at the same time.