开发者

Linux: need to design pre-fetcher to cache files from NAS into system memory

I am designing a server for the following scenario:

  1. a series of single images are stored on a NAS, lets say 100 of them
  2. a client connects to the server over TCP socket and requests image39
  3. server reads image39 from NAS and sends back to client over socket
  4. it is likely that the client will also request other images from the series, so:
  5. I would like to launch a thread that iterates through the images, reads them, and does a cat image39 > /dev/null to force cache into memory on server
  6. thread will fetch images as follows: image38, image40, image37, image41, etc.
  7. already fetched images are ignored
  8. if client now requests image77, I want to reset the fetch thread to fetch: image76, image78, etc.

This has to scale to many series and clients. Probably on the order of 1000 concurrent prefetches. I understand that threads can cause performance hit if there are too many. Would it be better to fork a new process instead? Is开发者_运维问答 there a more efficient way than threads or processes ?

Thanks!!!


This is premature optimization. Try implementing your system without tricks to "force" the cache, and see how it works. I bet it'll be fine--and you won't then need to worry about nasty surprises if it turns out your tricks don't play nice with other things on the system.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜