Implementing a thread queue/wait, how?

I have a timer calling a function every 15 minutes, this function counts the amount of lines in my DGV and starts a thread for each lines (of yet anoth开发者_Python百科er function), said thread parse a web page which can take anywhere from 1 second to 10 second to finish.

Whilst it does work fine as it is with 1-6 rows, anymore will cause the requests to time-out.

I want it to wait for the newly created thread to finish processing before getting back in the loop to create another thread without locking the main UI

                for (int x = 0; x <= dataGridFollow.Rows.Count - 1; x++)
                {
                    string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString();
                    int ID = int.Parse(getID);
                    Thread t = new Thread(new ParameterizedThreadStart(UpdateLo));
                    t.Start(ID);
                    // <- Wait for thread to finish here before getting back in the for loop
                }

I have googled a lot in the past 24 hours, read a lot about this specific issue and its implementations (Thread.Join, ThreadPools, Queuing, and even SmartThreadPool).

It's likely that I've read the correct answer somewhere but I'm not at ease enough with C# to decypher those Threading tools

Thanks for your time

to avoid the UI freeze the framework provide a class expressly for these purposes: have a look at the BackgroundWorker class (executes an operation on a separate thread), here's some infos : http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx http://msdn.microsoft.com/en-us/magazine/cc300429.aspx

Btw looks if I understand correctly you don't want to parallelize any operation so just wait for the method parsing the page to be completed. Basically for each (foreach look) row of your grid you get the id and call the method. If you want to go parallel just reuse the same foreach loop and add make it Parallel

http://msdn.microsoft.com/en-us/library/dd460720.aspx

What you want is to set off a few workers that do some task.

When one finishes you can start a new one off.

I'm sure there is a better way using thread pools or whatever.. but I was bored so i came up with this.

using System;
using System.Collections.Generic;
using System.Linq;
using System.ComponentModel;
using System.Threading;

namespace WorkerTest
{
    class Program
    {
        static void Main(string[] args)
        {
            WorkerGroup workerGroup = new WorkerGroup();

            Console.WriteLine("Starting...");

            for (int i = 0; i < 100; i++)
            {
                var work = new Action(() => 
                { 
                    Thread.Sleep(1000); //somework
                });

                workerGroup.AddWork(work);
            }

            while (workerGroup.WorkCount > 0)
            {
                Console.WriteLine(workerGroup.WorkCount);
                Thread.Sleep(1000);
            }

            Console.WriteLine("Fin");

            Console.ReadLine();
        }
    }


    public class WorkerGroup
    {
        private List<Worker> workers;

        private Queue<Action> workToDo;

        private object Lock = new object();

        public int WorkCount { get { return workToDo.Count; } }

        public WorkerGroup()
        {
            workers = new List<Worker>();
            workers.Add(new Worker());
            workers.Add(new Worker());

            foreach (var w in workers)
            {
                w.WorkCompleted += (OnWorkCompleted);
            }

            workToDo = new Queue<Action>();
        }

        private void OnWorkCompleted(object sender, EventArgs e)
        {
            FindWork();
        }

        public void AddWork(Action work)
        {
            workToDo.Enqueue(work);
            FindWork();
        }

        private void FindWork()
        {
            lock (Lock)
            {
                if (workToDo.Count > 0)
                {
                    var availableWorker = workers.FirstOrDefault(x => !x.IsBusy);
                    if (availableWorker != null)
                    {
                        var work = workToDo.Dequeue();
                        availableWorker.StartWork(work);
                    }
                }
            }
        }
    }

    public class Worker
    {
        private BackgroundWorker worker;

        private Action work;

        public bool IsBusy { get { return worker.IsBusy; } }

        public event EventHandler WorkCompleted;

        public Worker()
        {
            worker = new BackgroundWorker();
            worker.DoWork += new DoWorkEventHandler(OnWorkerDoWork);
            worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(OnWorkerRunWorkerCompleted);
        }

        private void OnWorkerRunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            if (WorkCompleted != null)
            {
                WorkCompleted(this, EventArgs.Empty);
            }
        }

        public void StartWork(Action work)
        {
            if (!IsBusy)
            {
                this.work = work;
                worker.RunWorkerAsync();
            }
            else
            {
                throw new InvalidOperationException("Worker is busy");
            }
        }

        private void OnWorkerDoWork(object sender, DoWorkEventArgs e)
        {
            work.Invoke();
            work = null;
        }
    }
}

This would be just a starting point.

You could start it off with a list of Actions and then have a completed event for when that group of actions is finished.

then at least you can use a ManualResetEvent to wait for the completed event.. or whatever logic you want really.

Call a method directly or do a while loop (with sleep calls) to check the status of the thread.

There are also async events but the would call another method, and you want to continue from the same point.

I have no idea why the requests would timeout. That sounds like a different issue. However, I can make a few suggestions regarding your current approach.

• Avoid creating threads in loops with nondeterministic bounds. There is a lot of overhead in creating threads. If the number of operations is not known before hand then use the ThreadPool or the Task Parallel Library instead.
• You are not going to get the behavior you want by blocking the UI thread with Thread.Join. The cause the UI to become unresponsive and it will effectively serialize the operations and cancel out any advantage you were hoping to gain with threads.

If you really want to limit the number of concurrent operations then a better solution is to create a separate dedicated thread for kicking off the operations. This thread will spin around a loop indefinitely waiting for items to appear in a queue and when they do it will dequeue them and use that information to kick off an operation asynchronously (again using the ThreadPool or TPL). The dequeueing thread can contain the logic for limiting the number of concurrent operations. Search for information regarding the producer-consumer pattern to get a better understand of how you can implement this.

There is a bit of a learning curve, but who said threading was easy right?

If I understand correctly, what you're currently doing is looping through a list of IDs in the UI thread, starting a new thread to handle each one. The blocking issue you're seeing then could well be that it's taking too many resources to create unique threads. So, personally (without knowing more) would redesign the process like so:

//Somewhere in the UI Thread
Thread worker = new Thread(new ParameterizedThreadStart(UpdateLoWorker));
worker.Start(dataGridFollow.Rows);

//worker thread
private void UpdateLoWorker(DataRowCollection rows)
{
   foreach(DataRow r in rows){
      string getID = r.Cells["ID"].Value.ToString();
      int ID = int.Parse(getID);
      UpdateLo(ID);
   }
}

Here you'd have a single non-blocking worker which sequentially handles each ID.

Consider using Asynchronous CTP. It's an asynch pattern Microsoft recently released for download. It should simplify asynch programming tremendouesly. The link is http://msdn.microsoft.com/en-us/vstudio/async.aspx. (Read the whitepaper first)

Your code would look something like the following. (I've not verified my syntax yet, sorry).

private async Task DoTheWork()
{
    for(int x = 0; x <= dataGridFollow.Rows.Count - 1; x++) 
    { 
        string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString(); 
        int ID = int.Parse(getID); 
        task t = new Task(new Action<object>(UpdateLo), ID); 
        t.Start();
        await t;
    }
} 

This method returns a Task that can be checked periodically for completion. This follows the pattern of "fire and forget" meaning you just call it and presumably, you don't care when it completes (as long as it does complete before 15 minutes).

EDIT
I corrected the syntax above, you would need to change UpdateLo to take an object instead of an Int.

For a simple background thread runner that will run one thread from a queue at a time you can do something like this:

    private List<Thread> mThreads = new List<Thread>();

    public static void Main()
    {       
        Thread t = new Thread(ThreadMonitor);
        t.IsBackground = true;
        t.Start();
    }

    private static void ThreadMonitor()
    {
        while (true)
        {
            foreach (Thread t in mThreads.ToArray())
            {
                // Runs one thread in the queue and waits for it to finish
                t.Start();
                mThreads.Remove(t);
                t.Join();
            }

            Thread.Sleep(2000); // Wait before checking for new threads
        }
    }

    // Called from the UI or elsewhere to create any number of new threads to run
    public static void DoStuff()
    {
        Thread t = new Thread(DoCorestuff);
        t.IsBackground = true;
        mActiveThreads.Add(t);
    }

    public static void DoStuffCore()
    {
        // Your code here
    }