how to crawl the web and get the list of sites that have some keywords

I am in need to scan through the entire web and get the links of the sites that have some keywords in my dictionary.

I need to get the URL's and others dynamically fetched from the web. Can anyone suggest me the proc开发者_如何学编程edure or the tool that can accomplish this for me.

Check with this:-

• Lucene/apache Solr for indexing.
• Perl LWP module LWP