Using observer pattern in the context of Qt signals/slots

I am trying to make some design decisions for an algorithm I am working on. I think that I want to use signals and slots to implement an observer pattern, but I am not sure of a few things.

Here is the algorithm I am working towards:

1.) Load tiles of an image from a large file
1a.) Copy the entire file to a new location
2.) Process the tiles as they are loaded
3.) If the copy has been created, copy the resulting data into the new file

so I envision having a class with functions like loadAllTiles() which would emit signals to tell a processTile() that another tile was ready to be processe开发者_开发问答d, while moving on to load the next tile.

processTile() would perform some calculations, and when complete, signal to writeResults() that a new set of results data was ready to be written. writeResults() would verify that the copying was complete, and start writing the output data.

Does this sound reasonable? is there a way to make loadAllTiles() load in a tile, pass that data somehow to processTile() and then keep going and load the next tile? I was thinking about maybe setting up a list of some sort to store the tiles ready to be processed, and another list for result tiles ready to be written to disk. I guess the disadvantage there is I have to somehow keep those lists in tact, so that multiple threads arent trying to add/remove items from the list.

Thanks for any insight.

It's not completely clear in your question, but it seems that you want to split up the work into several threads, so that the processing of tiles can begin before you finishing loading the entire set.

Consider a multithreaded processing pipeline architecture. Assign one thread per task (loading, copying, processing), and pass around tiles between tasks via Producer-Consumer queues (aka BlockingQueue). To be more precise, pass around pointers (or shared pointers) to tiles to avoid needless copying.

There doesn't seem to be a ready-made thread-safe BlockingQueue class in Qt, but you can roll-up your own using QQueue, QWaitCondition, and QMutex. Here are some sources of inspiration:

• Just Software Solutions' blog article.
• Java's BlockingQueue
• ZThreads's BlockingQueue

While there isn't a ready-made BlockingQueue within Qt, it seems that using signals & slots with the Qt::QueuedConnection option may serve the same purpose. This Qt blog article makes such use of signals and slots.

You may want to combine this pipeline approach with a memory pool or free list of tiles, so that already allocated tiles are recycled in your pipeline.

Here's a conceptual sketch of the pipeline:

TilePool -> TileLoader -> PCQ -> TileProcessor -> PCQ -> TileSaver -\
  ^                                                                 |
   \----------------------------------------------------------------/

where PCQ represents a Producer-Consumer queue.

To exploit even more parallelism, you can try thread pools at each stage.

You can also consider checking out Intel's Threading Building Blocks. I haven't tried it myself. Be aware of the GPL licence for the open source version.

Keeping the lists from corruption should be possible with any kind of parallelization lock mechanisms, be it simple locks, semaphores etc etc.

Otherwise, the approach sounds reasonable, even though I would say that the files had to be large in order for this to make sense. As long as they easily fit into memory, I don't see the point of loading them piecewise. Also: How do you plan on extracting tiles without reading the entire image repeatedly?