Will I need some kind of caching mechanism for this? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 11 years ago.

I've started building an application that part of which allows users to split-test and rotate offers, setting minimum and maximum amounts of traffic for a particular url, and a time to reset.

So, for example, one user might enter the url http://www.example.com, a minimum of 90 hits, a maximum of 150 and a time until reset of 1 hour. Every hour, a number will be chosen from between the two, and become the new hit amount for that hour. Once the the number of hits for http://www.example.com has been reached within an hour http://www.example.com will be removed from the list, until the next reset.

All in all, on a page load, at least 3-4 database queries are made at different points throughout.

Here are my questions:

Is this sustainable with large traffic amounts, say 2,500+ requests per hour without some kind of caching mechanism (which is almost impossible to build, because hit's need to be exact and updated on request, so I don't see how a cache could be built in this instance)?

What are the variables/factors that will determine whether or not this will be doable?

I realize my questions a little, "iffy", so please do ask clarification if you need it :)!

A开发者_如何学编程ny advice, comments, answers would be greatly appreciated!

Edit:

Wow, 4 close votes, yet no one bothers to post to say why. I've asked a relevant question, and while it may not be that well put together, I'd like to know how it doesn't adhere to the SO guidelines.

---

You may use something like APC or Xcache to keep the data in the RAM and you save everything via a crontab hourly

That would be a way to decrease the load and increase the speed of the website, but it may blow up your RAM if you got too much data to save. Also there's the risk of data loss if the data wasn't saved yet and you lose your RAM contents

If you only need to save +1/per click/per website which would be a simple array of the websites you observe and the number of clicks (probably a number between 90 and 150?), caching in RAM would be the best way to do it

Another way would be using files, which would also be faster than SQL and would have less load than constantly sending SQL queries

I hope I understood your question correctly

---

In my opinion there is too little information to decide whether you need caching here or not. On the one hand caching is seldom a bad idea (in some cases it can indeed be bad), on the other hand you might think, that it is not worth the development overhead. From my experience it depends a lot on what data is stored and how it is stored in the database. If you have for example one single table where you just fetch data from and use MyISAM table type, things can get incredible fast. If you have to JOIN and you decide to use InnoDB, things can get slower. It also depends a lot on the infrastructure you have available, etc. ... In my opinion there are too many open questions to decide whether caching is required here or not.

---

One way this can be solved is by turning Database itself into a cache.

E.g., say total amount of hits data (for 1 Hour) is 300 MB. You can allocate 600 MB to IO buffer of DB server.

That ways, data would almost always be "cached" in memory. This would give very good performance without adding a new component in the system. As and when new hits are recorded, that information would be recorded on disk.

But subsequent page hits will query it very often (2,500+ requests per hour) and DB server will decide to keep that row in memory (technically the page that contains the row) for a long time.

This way, DB server would itself perform job a cache.

Some hints on how to do this with PostgreSQL:

http://www.postgresql.org/docs/8.2/static/monitoring-stats.html