Storing 1000s of thumbnails

I have about 1 million thumbnails that I have on my harddisk and that is really increasing the latency on the disk.

Now how can i store those thumbnails on the harddrive to th开发者_运维百科at improve the performance and also fast fetching.

How do organisations like facebook, google images, etc store their images?

I have 10TB storage so that is not a problem. The problem is with the number of images.

Is this for a web application? If so, the thumbnails (and other stuff) might benefit from actually moving off your hard drive and onto a content delivery network. Large sites (as well as those trying to speed up content delivery) typically use content delivery networks (CDNs) to serve data that's mirrored all over the globe to ensure fast access regardless of where the request comes from.

I've used Rackspace CloudFiles and would recommend it, but this obviously isn't useful if you're looking to serve these images directly from the hard drive in question.

A RAID 0 would be an interesting idea... Though I doubt the usefulness of a million thumbnails.

Google & co store their information on multiple servers, which are most of the time backed-up by RAID and other fast-access/safekeeping methods. So that'd be your best bet :)

You should try Bildero image server - it allows to store and serve large amount of images. It has also some usefull features like dynamic resizing, custom watermarks, dynamic crop, zoom, spin, and many more - Bildero