How to download a html page via asp.net
how can i download a web page from my web app, then reading "title" and "开发者_JAVA百科description" metatag ? Like a web crawler, but in Asp.net and called by ..an asp.net web page ?
Thanks!
You can do a screen scrape of an external URL in .NET using the WebClient
class, which you'll find in the System.Net
namespace. Use the DownloadData
method to download the content from a specified URL. The downloaded data comes down as a byte array, but you can convert this to a string.
The following snippet shows how to use WebClient
to grab the HTML from my blog's homepage, http://scottonwriting.net/sowblog/default.aspx:
// Create a new WebClient instance.
WebClient myWebClient = new WebClient();
// Download the markup from
byte[] myDataBuffer = myWebClient.DownloadData("http://scottonwriting.net/sowblog/default.aspx");
// Convert the downloaded data into a string
string markup = Encoding.ASCII.GetString(myDataBuffer);
Once you have the markup you can use regular expressions or string searching methods to pick out the markup of interest.
Use the HTML Agility Pack and its HTMLWeb
class.
精彩评论