How to handle this (db) queue race condition?
Basically i have multi threads that adds data into a queue via SQLite. I have another one thread that pulls them and process them one at a time (too much resource to do multiple at once). The processing thread does this:
- pull data from DB
- foreach { proccess }
- if count == 0 { thread.suspend() } (waken by thread.resume())
- repeat
my wo开发者_JAVA百科rker thread does:
- Validates data
- Inserts into DB
- call Queue.Poke(QueueName)
When I poke it, if the thread is suspended I .resume()
it.
What I am worried about is if the process thread sees count==0
, my worker inserts and pokes then my process continues down the if and sleeps. It won't realize there is something new in the DB.
How should I write this in such a way that I won't have a race condition.
Processing thread:
- event.Reset
- pull data from DB
- foreach { proccess }
- if count == 0 then event.Wait
- repeat
And the other thread:
- Validates data
- Inserts into DB
- event.Set()
You'll have extra wakes (wake on an empty queue, nothing to process, go back to sleep) but you won't have missed inserts.
I think this may be the structure you need.
private readonly Queue<object> _Queue = new Queue<object>();
private readonly object _Lock = new object();
void FillQueue()
{
while (true)
{
var dbData = new { Found = true, Data = new object() };
if (dbData.Found)
{
lock (_Lock)
{
_Queue.Enqueue(dbData.Data);
}
}
// If you have multiple threads filling the queue you
// probably want to throttle it a little bit so the thread
// processing the queue won't be throttled.
// If 1ms is too long consider using
// TimeSpan.FromTicks(1000).
Thread.Sleep(1);
}
}
void ProcessQueue()
{
object data = null;
while (true)
{
lock (_Lock)
{
data = _Queue.Count > 0 ? _Queue.Dequeue() : null;
}
if (data == null)
{
Thread.Sleep(1);
}
else
{
// Proccess
}
}
}
精彩评论