cURL - Fetch data only if html page (link)

I am building a CSS Optimizer website, in this I will offer a option where a user submits a Website URL and on backend, server will parse all the anchors and link tags wi开发者_如何学Pythonthin that domain. So this will enable the site to concatenate + compress the css which may be unique on all different pages.

As all my website needs is anchor (html) and <link> what should I do to avoid fetching links that are non-html (like .doc, .pdf) etc

I don't think you can reply on just the extensions of targets. Clean URLs don't often have extensions at the end, content might be generated.

You should probably do an HTTP HEAD on every URL and then check the Content-Type which is returned.

What about checking the mime type of the related link with mime_content_type()?