Reverse Proxy for Static Files (hiding upstream filename/paths)

I've got a set of static files I want to serve. They're all available in an Amazon S3 bucket. However, I don't want to reveal the actual url/path of those files on S3. I only want a subset of those files to be available at any given time. The subset is determined dynamically by a Django application.

For example, let's say I have three files named /amazon/1.jpg, /amazon/2.jpg, and /amazon/3.jpg. I don't want to reveal their absolute URLs whatsoever. Instead I want the URL /image_a.jpg to dynamically map to开发者_StackOverflow社区 one of the three images and /image_b.jpg to dynamically map to another.

My envisioned solution is to have a reverse proxy (nginx or twisted) handle the URLs /image_a.jpg and image_b.jpg. However, I can't think of an easy way to dynamically map them to the appropriate/actual upstream file.

Can you guys help me come up with a scalable solution to this problem? (I want to be able to support 500+ concurrent accesses to these public URLs. Moreover, the upstream source files might be several MB).

Thanks for your help!

-Advait

You could do this with twisted's web server, although not sure how fast it would be. I've only used the twisted.web.wsgi.WSGIResource, so maybe this can be done simpler, but just set up an app like this:

http://twistedmatrix.com/documents/10.1.0/web/howto/using-twistedweb.html#auto19

And in your application, using the HTTP_REFERER in the environs argument map your URL any way you want and then use the twisted HTTP client to grab the content and serve it back as a response to the original request. Or maybe just use the regular python httplib if the async stuff gets a little complicated (you have to block the initial web request while waiting go grab the request from amazon).

Or you could do this with node.js's HTTP module in a few lines. Create an HTTP server and then for each request use the HTTP client request features in the module to serve back what you get from amazon.

Or you could use mod_proxy with apache and a regular expression and some rewrite rules.

Don't know how you'd do it with nginx, but if you can do it with apache httpd you can probably do it with nginx. I'd say going with node or a webserver like nginx probably makes the most sense.

With nginx: start by adding the proxy.conf file to you /etc/nginx/conf.d directory (see auxilliary file section: http://wiki.nginx.org/FullExample) Then use a similar configuration file for your server:

server {
    server_name mysver.com ;
    location /image_a.jog {
        proxy_pass http://$host/amazon/1.jpg ;
    }
    location /image_b.jog {
        proxy_pass http://$host/amazon/2.jpg ;
    }
}

The server section will answer myserver.com/image_a.jpg with myserver.com/amazon/1.jpg without revealing the real url. (you can replace $host by what ever you want e.g. http://127.0.0.1/foo/amazon/1.jpg