multiple writes on a writestream in node.js
I've been looking at the code of node-dirty and noticed that when writing a lot of data to a file, the original programmer has chosen to bunch the writes into several groups and issue writes of the groups one at a time, but they are all issued simultaneously as part of one loop, without waiting for any callbacks. I have three questions about this. I have a similar problem to solve.
- Is this more efficient in some way? Should I be bundling writes too?
- How should I choose the optimum bundle size? Why not just write one group?
- If I sign up to the on('drain') event on the writestream, will it be emitted only once after all the simultaneou开发者_StackOverflow社区sly issued writes have completed? Or after each? (my guess is the former)
- If the on('error') is emitted, will the ('drain') event also be emitted? Or are they mutually exclusive?
thanks
Is this more efficient in some way? Should I be bundling writes too?
It's inefficient to make many small writes. sending a write command has overhead attached to it. So writing just 5 bytes instead of a 1000 is more expensive.
How should I choose the optimum bundle size? Why not just write one group?
Optimum size sounds like a black art to me. I presume there are good reasons for not making it one big write. Probably to start writing earlier then later. It's slightly more efficient to start a bit earlier.
If I sign up to the on('drain') event on the writestream, will it be emitted only once after all the simultaneously issued writes have completed? Or after each? (my guess is the former)
Drain triggers when everything in the write queue has finished writing. So as long as you append to the write queue faster then it writes it, it should only trigger once. You'd need one hell of a system to pull of an edge-case like that.
If the on('error') is emitted, will the ('drain') event also be emitted? Or are they mutually exclusive?
Even if it is emitted it doesn't make sense to do error handling in 'drain'. If an error has occurred I would always assume that the entire writing operation has failed and not try to recover mid-write.
For 4. If the on('error') is emitted, will the ('drain') event also be emitted? Or are they mutually exclusive?
You are worried about it since you don't want to maintain state in your application right. So, maybe you could use a convenience function:
function not_if(proc, veto_inner) {
var vetoed = false;
return {
proc: function() {
if (!vetoed) { return proc.apply(null, arguments); }
},
vetoer: function() {
if (!vetoed) {
vetoed = true;
veto_inner.apply(null, arguments);
}
};
}
Now, you can set the 'error' handler to vetoer and the 'drain' handler to 'proc' and not wirry about 'drain' being called after 'error' is called.
精彩评论