Multiple data sources: data storage and retrieval approaches

I am building a website (probably in Wordpress) which takes data from a number of different sources for display on various pages.

The sources:

• A Twitter feed
• A Flickr feed
• A database on a remote server
• A local database

From each source I will mainly retrieve

• A short string, e.g. for Twitter, the Tweet, and from the local database the title of a blog page.
• An associated image, if one exists
• A link identifying the content at its source

My question is:

What is the best way to a) store the data and b) retrieve the data

My thinking is:

i) Write a script that is run every 2 or so minutes on a cron job

ii) the script retrieves data from all sources and stores it in th开发者_开发百科e local database

iii) application code can then retrieve all data from the one source, the local database

This should make application code easier to manage - we only ever draw data from one source in application code - and that's the main appeal. But is it overkill for a relatively small site?

I would recommend putting the twitter feed and flickr feed in JavaScript. Both flickr and twitter have REST APIs. By putting it on the client you free up resources on your server, create less complexity, your users won't be waiting around for your server to fetch the data, and you can let twitter and flickr cache the data for you.

This assumes you know JavaScript. Once you get past JavaScript quirks, it's not a bad language. Give Jquery a try. JQuery Twitter plugin Flickery JQuery plugin. There are others, that's just the first results from Google.

As for your data on the local server and remote server, that will depend more on the data that is being fetched. I would go with whatever you can develop the fastest and gives acceptable results. If that means making a REST call from server to sever, then go for it. IF the remote server is slow to respond, I would go the AJAX REST API method.

And for the local database, you are going to have to write server side code for that, so I would do that inside the Wordpress "framework".

Hope that helps.