Dynamic creation of Consumer

I have implemented a java version of Producer && Consumer pattern in java and I have noticed that my solution is an static way !!

Precicely I have the code in the main test like where i have 1 Producer(read a chunk of a file and put it to a buffer) and many* consumer (from a buffer keeps chunks ) in the example 4:

 Producer master=开发者_运维百科 new Producer();

 Consumer one= new Consumer();
 Consumer two= new Consumer();
 Consumer three= new Consumer();
 Consumer four= new Consumer();

 master.start();
 one.start();
 two.start();
 three.start();
 four.start();

And let's ROCCccccck..

So it's all ok but, I would Like implement a dynamic way,in which the number of consumers are established before all starts... IF I compute the java code ??:

long usableFreeMemory= Runtime.getRuntime().maxMemory()-Runtime.getRuntime().totalMemory()
        +Runtime.getRuntime().freeMemory();

I obtain the free memory of JVM ?? or the free memory of the computer ( RAM) ??

After that i obtain a free memory how I could know how many Consumer (threads) I needs to compute the parsing of a huge file?? Which Math formula a I could use to create the necessary threads??

thanks..

As a strategy for the number of consumers to create I would suggest you inspect Runtime.availableProcessors(). This method will tell you "... the number of processors available to the Java virtual machine." (from the JavaDoc). I submit this as the best approach because if you have more consumers than available processors you any consumer past that point will have to wait for another to enter a sleep state before it can being, and therefore you effectively have numProcessors consumers anyway.

for(int i=0;i<Runtime.numProcessors();i++){
  new Consumer().start();
}

Of course if your program is distributed you could start a consumer for each processor on each machine.

With your memory checks you obtain the memory of the JVM, so that´s good. Loading the memory values of the Operating System will not make sense for your purpose.

Is the consumer always doing the same, or will it just execute any Task / Runnable it gets supplied? If it's always the same then you could benchmark how much memory each consumer needs and based on that calculate an amount of Threads. Otherwise you would need to check dynamically what memory each Thread uses and depending on the free space in the JVM decide if it's wise to start another Thread.

Of course it's not only about memory, but also about CPU. So you might have enough memory to spawn new Threads, but if the CPU has no more free resources then you will get performance issues.

You can also have a look at the Java Excecutor interface and it's implementations, they might come in very handy for a producer / consumer pattern.

If you wish to create a dynamic number of Producers or Consumers, why don't you just add command line arguments to your program to specify their respective numbers, and parse those? I wouldn't try to use some heuristic based off of free memory, as this may vary on various platforms.

Once you know how many consumers you want you can put them in an array

int consumerCount = 
Consumer[] consumers = new Consumer[consumerCount];
for(int i=0;i<consumerCount;i++)
    consumer[i] = new Consumer();
for(Consumer consumer: consumers) consumer.start();

for(Consumer consumer: consumers) consumer.stop();