WebException.Response.GetResponseStream() limited to 65536 characters

I am trying to retrieve HTML code from a webpage using HttpWebRequest and HttpWebResponse.

response = (HttpWebResponse)request.GetResponse();
...
Stream stream = response.GetResponseStream();

The response object has a ContentLength value of 106142. When I look at the stream object, it has a length of 65536. When reading the stream with a StreamReader using ReadToEnd(), only the first 65536 characters are returned.

How can I get the whole code?

Edit:

Using the following code segment:

catch (WebException ex)
{
    errorMessage = errorMessage + ex.Message;
    if (ex.Response != null) {
 开发者_运维问答       if (ex.Response.ContentLength > 0) 
        {
            using (Stream stream = ex.Response.GetResponseStream())
            {
                using (StreamReader reader = new StreamReader(stream))
                {
                    string pageOutput = reader.ReadToEnd().Trim();

ex.Response.ContentLength = 106142

ex.Response.GetResponseStream().Length = 65536

stream.Length = 65536

pageOutput.Length = 65534 (because of the trim)

And yes, the code is actually truncated.

You can find an answer in this topic in System.Net.HttpWebResponse.GetResponseStream() returns truncated body in WebException

You have to manage the HttpWebRequest object and change DefaultMaximumErrorResponseLength property. For example :

HttpWebRequest.DefaultMaximumErrorResponseLength = 1048576;

ReadToEnd does specifically just that, it reads to the end of the stream. I would check to make sure that you were actually being sent the entire expected response.

There seems to be a problem when calling the GetResponseStream() method on the HttpWebResponse returned by the exception. Everything works as expected when there is no exception.

I wanted to get the HTML code from the error returned by the server.

I guess I'll have to hope the error doesn't exceed 65536 characters...