How can I get If-Modified-Since and If-None-Match headers in IIS when using some sort of wildcard mapping?

I have an image server, that takes URLs of the form

http://www.myserver.com/photo/foo/bar/baz/~120x120/crop/abc123.jpg

and dynamically generates thumbnails based on the parameters parsed from the URL string. We've had this ru开发者_开发百科nning in two configurations - one by mapping the 404 error in IIS to a .aspx page, with the image processing in the code-behind, and a second by implementing IHttpHandler and adding a line to the system.web section of /photo/web.config like so:

<httpHandlers>
    <add verb="GET,HEAD" path="*" type="App_Code.ThumbnailHandler" />
</httpHandlers>

I need to add some support for HTTP caching, but in both configurations, IIS is silently stripping the If-Modified-Since and If-None-Match headers - Fiddler shows the browser sending them to the server, but then they're gone by the time my page or handler gets access to the Request.Headers collection.

How can I write a piece of code that handles arbitrary, wildcard URL requests, with ANY file extension, but can still access the headers I need to be able to respond with a 304 Not Modified if the client already has a current version of the resource? My next thing to try would be rewriting it as an MVC application but that really seems like overkill.

EDIT: OK, what's REALLY crazy is that if I hit

http://www.myserver.com/photo/this/is/a/random/string/of/gibberish.aspx

(i.e. with .ASPX file extension) - the headers are INTACT! But if I hit

http://www.myserver.com/photo/this/is/a/random/string/of/gibberish.jpg

They're missing... any ideas?

The only workaround I've found for this is to implement IHttpHandler as described above, but then map every file extension you're 'matching' (in our case .JPG, .PNG and .TIF) to %windir%\Microsoft.NET\Framework\v2.0.50727\aspnet_isapi.dll in IIS. That, for some bizarre reason, means IIS will invoke your HTTP handler without mangling the headers first.