Programmable WebCrawler with C#

I would like to extract 开发者_JS百科specific data form a known Url : from html tags like span, a, divs ... !

So I'm searching for an existing library for a WebCrawler written in C# ! ... or maybe any other idea.

Thank you !

For parsing HTML, you might want to look into the HTML Agility Pack. As for grabbing the HTML from a website, you can use WebClient.

This guy wrote an article on using HtmlAgilityPack for web crawling from C#, and this is a framework for web crawling, though the project seems to be dead, you can look at them and see if they fit your needs.

A lot of people on SO like HtmlAgilityPack.