Super slow HttpWebRequest

I'm making a website scraper for a project I'm doing. I've gotten everything to work great, however, loading the actual page takes F-O-R-E-V-E-R. You can see the page it's loading here: MCServerList.Net

Here is the code I am using:

private CONST REQUESTURL as string = "http://www.MCServerList.net/?page="
private chunkId as int32 = 1

Dim req As HttpWebRequest = WebRequest.Create(REQUESTURL & chunkId)
Dim res As HttpWebResponse = req.GetResponse()
Dim Stream As Stream = res.GetResponseStream()

I开发者_Go百科 then use "Stream" and load it through the HTMLAgilityPack found free online. It loads the page quickly, however, the initial request usually takes ~20-30 seconds.

Any help would be appreciated!

I just ran the following code and ignoring the first initial compile I average about 3.3 seconds for GetResponse() and 0.2 more seconds for Load(). Are you on a fast connection? Are you sure this is where the bottleneck is?

Option Explicit On
Option Strict On

Imports System.Net

Public Class Form1

    Private Const REQUESTURL As String = "http://www.MCServerList.net/?page="
    Private chunkId As Int32 = 1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim ST As New System.Diagnostics.Stopwatch()
        ST.Start()
        Dim req = WebRequest.Create(REQUESTURL & chunkId)

        Dim res = req.GetResponse()
        Trace.WriteLine(String.Format("GetResponse() : {0}", ST.Elapsed))
        Using Stream As System.IO.Stream = res.GetResponseStream()
            Dim X As New HtmlAgilityPack.HtmlDocument()
            X.Load(Stream)
        End Using
        Trace.WriteLine(String.Format("Load() : {0}", ST.Elapsed))
        ST.Stop()
    End Sub
End Class