Live activity stream architecture

I have posted earlier on a similar question but I would like to know what is the underlying technology to make an activity feed work in real-time? I am designing it 100% dat开发者_如何学运维abase-driven in MySQL but running into issues are there are 200+ activities to stream out and there are different types of streams + need to support at least 500,000 concurrent users to start with. There are geographic-based streams, network streams, friend streams, business steams, etc. All stream data is self-hosted based on activities on my site. My platform is Linux, MySQL, PHP.

Issues facing: 1) Unsure about what way to approach this. Should I just do it in AJAX, PHP, and MySQL or is this done via RSS/XML or do I need to use old fashion read from text files? 2) How does the stream update in real-time - store activities client-side in a cookie or session, server to client push, client to server pull, etc? 3) Is it more server related like do I need a specialized server just to do this, assuming I need to support 500,000 concurrent users? 4) If I need to use specialized frameworks for this are there any open-source frameworks?

Any links to sample architecture/implementation strategies or resources would be helpful.

I've got no idea what you mean specifically by "activity stream"; and I haven't done anything like this before, but here's what I'd be thinking:

• Serving 50K users is probably going to mean multiple servers - so you can't use or do anything that is going to introduce server affinity (using session variables, etc).
• You want to be as efficient as you can across the board - minimal data exchanges (both in size and frequency).
• Avoid unnecessary parsing (e.g: XML) or operations that are expensive; dealing with big strings, etc.
• Document carefully, performance test often. You might want to start with some Proof of Concepts.
• Use memory: reading and writing to disk is expensive, so shunt data into memory and deal with it there; for example, on application start-up, you might shunt the whole database (well, all the tables/data that you'll need to serve requests) into memory so you don't need to establish database connections for every request. This doesn't stop you from doing asynchronous write-backs to the database (having a server die and losing in-memory data tends to suck).
• Investigate mature caching technologies.
• By asynchronous where you can - you want minimal bottle-knecks.