开发者

Potential problems with a multithreaded, computation intensive application

I have been working on a project which could be summarized as "a computation intensive analysis tool" and I chose to write the code so that multiple threads are used to process several individual calculations in parallel.

After a series of problems due to lack of experience in programming multithreaded projects, it appears that I have finally managed to get it to work like I have intended. Calculations are picked one by one from a queue, and processed simultaneously. Following a particular advice (regarding appropriate number of threads) I got in response to a previous question I have set it up so that there's one thread populating the process queue, while the rest of the threads in the pool pick up processes from the queue in succession. On my i7 machin开发者_如何学Pythone with 8-cores I have gone with 1 producer + 8 consumer = 9 threads.

Now my question is as follows:

It's no surprise that when everything works as intended, the CPU usage of the machine hits the roof. I have been following the CPU usage through Gnome System Monitor, and it seems like the cores are on 90-100% load during the analysis stage of the software. Other than slowing down other programs (e.g. browser, mail, pdf reader etc) that are running at the same time, are there any other potential problems I should be aware of? Could having a high load create any artifacts for instance in JVM?

Considering that my software is intended to run as a scientific analysis tool rather than regular desktop application, slowing down other programs is something I am willing to accept, but I would like to know if there are other problems I should be aware of. Any practical examples or previous experience is very welcome.

Thanks in advance,


If it is a computationally intensive task, then I have found it useful to run such tasks at background priority. Unless the task is memory intensive and causes massive paging, this should allow other processes to run largely unhindered while still allowing the computational process to get the lion's share of actual available CPU. If you are running on Windows, then sometimes the GUI becomes unresponsive and it can become difficult to kill the process or get other work done. (Unix/Linux is better in this regard, but it can still be irritating.)

You really want to ensure that you don't use so much memory that your machine starts swapping like mad.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜