pratical challenge: retrieving cookies from this site in .Net

I have a pratical qu开发者_Python百科estion here. I have to access to a site and get the cookies information to reuse it in subsequent navigation. I don't have any browser control on my side, because everything is supposed to run on a server. With some site the task is pretty easy, but there are some site that send back cookies in a way I cannot manage to intercept.

On example is Italian highways site: I can't understand how they sent back cookies and most important, how can I capture those cookies from a .Net application.

I've tried both WebClient object, looking into header, in this way:

var wc = new WebClient();
var htmlMainPage = wc.DownloadString(new Uri(AutostradeMainSite));
string cookies = wc.ResponseHeaders["Set-Cookie"];

but I don't get any result. Even by looking into header, there's no cookies there.

Then I tried with HttpWebRequest object, but I wasn't able to retrieve the cookies:

HttpWebRequest wr = (HttpWebRequest)HttpWebRequest.Create(AutostradeMainSite);
wr.Method = "GET";
HttpWebResponse response = (HttpWebResponse)wr.GetResponse();
var cookies = response.Cookies

What am I doing wrong?

Analyzing the main page of the site with some developer tools for IE or Chrome, I can see that some cookies are sent to the browser, but I cannot see them neither in the header, nor in the javascript... How can it work? Thanks in advance for any help.

The WebClient class can't handle cookies out of the box, although it's pretty easy to create a derived version that supports cookies. You can do it with the HttpWebRequest class, but you need to set the CookieContainer property:

HttpWebRequest req = WebRequest.Create("http://www.google.com") as HttpWebRequest;
req.CookieContainer = new CookieContainer();
HttpWebResponse res = req.GetResponse() as HttpWebResponse;
CookieCollection cookies = res.Cookies;

The site may be detecting that you're using an unknown "browser", and assuming you cannot accept cookies, so it doesn't even provide one. The answer would be to craft a custom WebRequest that looks like it comes from IE 8 instead of the .NET WebClient. You should be able to do this by setting a value for the User-Agent key of WebClient.Headers.