开发者

WebClient.DownloadFile in BackgroundWorker performance unacceptable

I have a WinForms application that needs to download many (tens of thousands) fairly large (multi-MB) files per day. I wrote a simple test using the following code:

using (var wc = new System.Net.WebClient())
{
    foreach (string url in UrlsToDownload())
    {
        string targetPath = SafeFilePathOf(url);
        wc.DownloadFile(url, targetPath);
    }
}

The target machine has a 1 Gb/s connection, but testing shows a sustained download of about 1MB/s. This is less than what I was expecting, but the source 开发者_运维问答servers may have slower connections. A full day's download will require several hours of connectivity, which is acceptable. Network utilization is fairly constant at about 1%:

WebClient.DownloadFile in BackgroundWorker performance unacceptable

However, I need to perform downloads in a background thread and support cancelling and download progress. The .Net System.ComponentModel.BackgroundWorker seems designed for this, so I put the exact same code in a BackgroundWorker instance and call RunWorkerAsync:

WebClient.DownloadFile in BackgroundWorker performance unacceptable

Download performance plummets to about 0.05 MB/s. A day's work will require about a week to perform; this is not going to fly.

Why is BackgroundWorker performance so bad? Neither the CPU nor network is overloaded. The application is not blocked, I simply moved the download code from the UI thread to a BackgroundWorker. Calling backgroundWorker.Priority = ThreadPriority.AboveNormal has no effect.


Increase the connection limit on the WebClient:

internal class CustomWebClient : WebClient
{
    public CustomWebClient()
    {
    }

    protected override WebRequest GetWebRequest(Uri uri)
    {
        HttpWebRequest w = (HttpWebRequest)base.GetWebRequest(uri);

        w.ServicePoint.ConnectionLimit = 100;
        return w;
    }
}


Network activity performs same whether it's background thread or UI thread. The problem can be any other code of reporting progress or logging activity etc. That may block the thread.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜