Get info from multiple files, match it and then display to end user, what is fastest?

I need to build a website where we display data that is refreshed every 5minutes in a text file with a | separator.

I currently use Java to do this.

What I do now: I grab the textfile for every request through the website and process it and then display the data to the end user, this works fine, since J开发者_如何学Goava can go through like 5000 lines of data fast, and when I filter it it is still extremely fast.

However now the management wants the following: They added 3 textfiles with the | separator to it, and now want me to also read those files and match the information on certain fields, and if there is a match also display that information to the end user.

I think soon enough, although Java is fast, I will run into trouble when 10 people want that information and I have to run through 4 total files matching the information.

What can I do to make this process super fast?

My Creative solutions so far:

-Leave it this way, since Java is fast and end users can wait (probably less then 1second)

-Have a background process that dumps new data into a MySQL database every 5minutes, since databases are extremely good at getting the same data from multiple tables.

Thank you!

Both solutions are good. A few things to keep in mind:

• If you're going to keep the user waiting for more than 1 second, consider making a landing page and showing a "loading info - please wait" message while your page loads the information (using AJAX, for example).
• Chances are management will add more files later. Coding a database solution now might save you from hacking it now and having to do it the right way later anyway.
• Other reasons to use the database are the DBMS extra advantages in caching, locking, joining...

I'm not really a Java guy, but I'm pretty sure you should be able to read these files into cache for 5 minutes... this way the data is combined and stored in memory... In addition to putting it in memory, you could store it in a hash table or some other data structure that makes searching faster (find value versus evaluate each value)...