Making sense of x-* headers when caching facebook images in desktop java app

This should be a fairly simple one... as is so often the case...

I'm writing a desktop application in Java, which must keep a cache of images from Facebook.

I don't seem to be able to reliably determine when I have a cache hit or miss as I would expect to based on the resource modification date.

I've noticed that the last-modified header in facebook's responses is almost always (in the case of profile pictures as retrieved by the method below) midnight on 1st Jan 2009... Browsing through a few pages on the site with Chrome's resource monitor open, I see that many images do indeed have this date set as the last-modified, and that in cases where this is true they also at least tend to have X-Backend, X-Blockid & X-Cache-by (or sometimes X-N) fields. There are other cases in which last-modified is set to some older date and X-* are not present, and other permutations, too (mostly it seems on interface and ad graphics).

Thus far, searching for info on these x-* headers, combined with limited basic knowledge of HTTP, has left me little the wiser as to how to properly implement my cache. Clearly between an ordinary browser and the facebook servers, there is no trouble ascertaining whether or not the images in cache are up to date, but it is not clear to me the mechanism that is being used to determine this.

At the moment, I have a method that opens a URLConnection to the appropriate facebook resource, setting ifmodifiedsince in cases where there already exists a corresponding file in the cache. If I get a 304 response code, then I return - after calling setLastModified on the cache file for good measure (partly as, for now, the cache of images is going into SVN & they all get lastModified set to whenever they happened to be checked out).

public static void downloadPicture(String uid) {
    try {
        URL url = new URL("http://graph.facebook.com/" + uid + "/picture?type=large");
        String fileName = "data/images/" + uid + ".jpg";

        HttpURLConnection conn = (HttpURLConnection) url.openConnection();

        File cachedPic = new File(fileName);
        long lastModified = cachedPic.lastModified();
        if (cachedPic.exists()) {
            conn.setIfModifiedSince(lastModified);
        }

        conn.connect();

        if (conn.getResponseCode() == 304) {
            System.out.println("Using cache for " + uid);
            cachedPic.setLastModified(conn.g开发者_开发知识库etLastModified());
            return;
        }

        System.out.println("Downloading new for " + uid);

        ReadableByteChannel rbc = Channels.newChannel(conn.getInputStream());

        FileOutputStream fos = new FileOutputStream(fileName);
        fos.getChannel().transferFrom(rbc, 0, 1 << 24);
        fos.close();
    } catch (MalformedURLException e) {
        System.err.println("This really shouldn't happen...");
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

There are undoubtedly other improvements that should be made to the way I do this process (the code above takes an appreciable amount of time for each individual item, which it clearly shouldn't and I'm sure I can resolve - although comments on best practice are still welcome), but what I need to find out is why I seem to always get 304 when there is any cached file present, and what's the correct method for ensuring that the proper requests are sent.

I think part of the problem here is the redirect that FB does when using the graph API URL to get the picture. The URL http://graph.facebook.com/user_uid/picture will actually redirect you to another URL for the picture. From what I have seen, the redirected URL changes when you change the profile picture. So, the actual picture URL would not change very often, the URL that you get redirected to would change.

I cannot point to any "official" documentation to back up the claims, this is just what I have experienced.