How to get plaintext from the response of a WebRequest class in C#
I want to get plain text using WebRequest class, just like what we get when we use webbrowser1.Document.Body.InnerText
. I have tried the following code
public string request_Resource()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.开发者_开发知识库Create(myurl);
Stream stream = request.GetResponse().GetResponseStream();
StreamReader sr = new StreamReader(stream);
WebBrowser wb = new WebBrowser();
wb.DocumentText = sr.ReadToEnd();
return wb.Document.Body.InnerText;
}
when i execute this is get a NullReferenceException
.
Is there a better way to get a plain text.
Note: I cannot use webbrowser control directly to load the webpage, because, i don't want to deal with all those events that fire up multiple times when ever a page is loaded.
UPDATE: I have changed my code to use WebClient Class instead of WebRequest upon suggestion My code looks something like this now
public string request_Resource()
{
WebClient wc = new WebClient();
wc.Proxy = null;
//The user agent header is added to avoid any possible errors
wc.Headers.Add("user-agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 ( .NET CLR 3.5.30729; .NET4.0C)");
return wc.DownloadString(myurl);
}
I am considering using HTML Utility Pack, can anyone suggest any better alternative.
You're looking for the HTML Agility Pack, which can parse the HTML without IE.
It has an InnerText
property.
To answer your question, you need to wait for the browser to parse the text.
By the way, you should use the WebClient
class instead of WebRequest
.
Use webclient:
public string request_Resource()
{
WebClient wc = new WebClient();
byte[] data = wc.DownloadData(myuri);
return Encoding.UTF8.GetString(data);
}
This will give you the content of the website. Then you can use HtmlAgilityPack to parse the result.
If you need just plain HTML text, then you have already wrote that code.
public string request_Resource()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(myurl);
Stream stream = request.GetResponse().GetResponseStream();
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
精彩评论