How to save webpage to string with cookies support (httpWebRequest)

I need to read webpage and store its content in string for further processing.

Sounds simply but I have problem with cookies support.

Opened page says I need browser supporting cookies (or turned on).

I've made method trying do that via httpWebRequest - which normally works to me but I've come to a standstill with those unfortunate cookies...

Any idea how to make it working?

Here is my method:

string ReadHtml (string address, string encoding) {
   Uri url = new Uri(address);

   CookieContainer cookieContainer  = new CookieContainer();
            HttpWebRequest httpWebRequest  =     (HttpWebRequest)HttpWebRequest.Create(url);
   httpWebRequest.AllowAutoRedirect = true;
            httpWebRequest.KeepAlive   = true;
            httpWebRequest.CookieContainer  = cookieContainer;
            httpWebRequest.UserAgent   = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
            httpWebRequest.Method    = "GET";
            HttpWebResponse webResponse = (HttpWebResponse)httpWebRequest.GetResponse();

   // Code Page
   Encoding enc = Encoding.GetEncoding(encoding);  

   // Read content
   StreamReader loResponseStream = new     StreamReader(we开发者_如何学GobResponse.GetResponseStream(),enc);
   string   lcHtml    = loResponseStream.ReadToEnd();

   webResponse.Close();
   loResponseStream.Close();

   return lcHtml;

  }

The page (until you r making request) sends you a cookie and at another request this page is trying to read thatcookie. When you don't provide cookie, then page think that your browser desn't support that mechanism.

I recommend you to:

1. Open Fiddler application
2. Check what cookies are saved (trying to save)

The solution is to provide back to page exacly that cookie, that this page sends you.

I'm not sure I quite understand the question... are you talking about taking the contents of a webpage and storing them into a cookie? Although I wouldn't exactly recommend this, if you need to, use:

HttpCookie cookie = new HttpCookie("cookieName");
cookie.Value = ReadHtml("http://www.google.com/", "utf-8");
cookie.Expires = DateTime.Now.AddHours(1);
Response.Cookies.Add(cookie);