C# - Connection: keep-alive Header is Not Being Sent During HttpWebRequest

I'm trying to send to send the following header with my HttpWebRequest:

Connection: keep-alive

However, the header is never sent. Fiddler2 is showing that whene开发者_C百科ver I request the page in Google Chrome, the header is sent. However, my application refuses to send this header for some reason.

I have set the KeepAlive property to true (it's true by default anyway), yet the header still does not get sent.

I am trying to send this header with multiple HttpWebRequests, but they all basically look like this:

HttpWebRequest logIn6 = (HttpWebRequest)WebRequest.Create(new Uri(responseFromLogIn5));
logIn6.CookieContainer = cookies;
logIn6.KeepAlive = true;
logIn6.Referer = "https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/";
logIn6.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1";
logIn6.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
logIn6.Headers.Add("Accept-Encoding:gzip,deflate,sdch");
logIn6.Headers.Add("Accept-Language:en-US,en;q=0.8");
logIn6.Headers.Add("Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3");
logIn6.AllowAutoRedirect = false;

HttpWebResponse logIn6Response = (HttpWebResponse)logIn6.GetResponse();
string responseFromLogIn6 = logIn6Response.GetResponseHeader("Location");

cookies.Add(logIn6Response.Cookies);

logIn6Response.Close();

Does anyone know what I have to do to make sure this header is sent?

Fiddler2 Raw From Chrome:

GET xxx HTTP/1.1
Host: accounts.google.com
Connection: keep-alive
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: xxx

HTTP/1.1 302 Moved Temporarily
Set-Cookie: xxx
Set-Cookie: xxx
Location: xxx
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Sat, 17 Sep 2011 22:27:09 GMT
Expires: Sat, 17 Sep 2011 22:27:09 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 2176
Server: GSE

Fiddler2 Raw From My Application:

GET xxx HTTP/1.1
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Host: accounts.google.com

HTTP/1.1 302 Moved Temporarily
Location: xxx
Content-Type: text/html; charset=UTF-8
Date: Sun, 18 Sep 2011 00:05:40 GMT
Expires: Sun, 18 Sep 2011 00:05:40 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 573
Server: GSE

I'm trying to get the second Fiddler2 raw information to look like the first Fiddler2 raw information.

I've had the same issue: The Connection: Keep-Alive header is not sent except the first request, and the server I accessed won't give me the correct response if it is missing. So, here are my workarounds to this issue:

First is set the ProtocolVersion property of HttpWebRequest instance to HttpVersion.Version10. Except the http command will become GET xxx HTTP/1.0, it works and uses only the public API.

The second way uses the reflection to modify the internal property ServicePoint.HttpBehaviour of HttpWebRequest instance, like this:

var req = (HttpWebRequest)WebRequest.Create(someUrl);

var sp = req.ServicePoint;
var prop = sp.GetType().GetProperty("HttpBehaviour", 
                        BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);

req.GetResponse().Close();

Hope this helps.

I struggled with this problem for half a day! And dear old Fiddler (my guardian angel) was inadvertantly part of the problem:

Whenever I tested my HTTP POSTs using Fiddler monitoring ON - the problem DIDN'T appear Whenever I tested my HTTP POSTs with Fiddler monitoring OFF - the problem DID appear

My POSTS were sent with protocol 1.1 and the Keep-Alive was ignored/redundant/why after the initial connection. i.e. I could see it in the header of the first POST (via Fiddler!), but not in subsequent POSTs despite using the same code. Hey ho ...

But the remote server would only respond if Keep-Alive was sent. Now I can't prove this, but I suspect that Fiddler monitoring the connection caused the remote server to think or believe that the connection was still active (despite no Keep-Alives sent after my first POST) and responded correctly. As I said, the second I turned Fiddler off, the absence of Keep-Alives caused the remote server to timeout on me..

I implemented the 1.0 solution described above and my POSTS now work, with or without Fiddler on or off. Hope this helps somebody else stuck somewhere ...

You doing it right. The code should result in following header added:

Connection: Keep-Alive

Post the code that you use for sending request and Raw output from Fiddler if you don't see this header. You may also ignore this because HTTP 1.1 connection is keep-alive by default.

Update: it looks like .NET only sets Keep-Alive explicitly for the first (!) request. Further requests to the same host/url will not have this header presumably because underlying tcp connection is already being reused.

After downloading HttpWebRequest source code, noticed that every property checks some known headers for HeaderCollection. To get rid of that doing some reflection stuff on that collection make it work

var webRequest = (HttpWebRequest) WebRequest.Create(url);
webRequest.Headers.GetType().InvokeMember("ChangeInternal",
    BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.InvokeMethod,
    Type.DefaultBinder, webRequest.Headers, new object[] {name, value}
);

I know the answer for this as I had the same problem and managed to solve it by inheriting the webclient and overriding it's Get Web Request method.

See the code below:

    public class CookieAwareWebClient : WebClient
{
    public CookieContainer CookieContainer { get; set; }

    public CookieAwareWebClient()
        : this(new CookieContainer())
    { }

    public CookieAwareWebClient(CookieContainer c)
    {
        this.CookieContainer = c;
    }

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest request = base.GetWebRequest(address);
        var castRequest = request as HttpWebRequest;

        if (castRequest != null)
        {
            castRequest.KeepAlive = true; //<-- this what you want! The rest you don't need. 
            castRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
            castRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36";
            castRequest.Referer = "https://www.jobserve.com/gb/en/Candidate/Login.aspx?url=48BB4C724EA6A1F2CADF4243A0D73C13225717A29AE8DAD6913D";
            castRequest.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
            castRequest.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
            castRequest.CookieContainer = this.CookieContainer;
        }

        return request;
    }
}

As you can see I am not only enabling keep-alive but I am utilizing cookies and other headers also!

I hope that helps!

Kiran